[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:59:04 dist-test-slave-07c2 krb5kdc[4434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:59:04 dist-test-slave-07c2 krb5kdc[4434](info): set up 2 sockets
Aug 09 00:59:04 dist-test-slave-07c2 krb5kdc[4434](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 00:59:05.337661 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 00:59:05 dist-test-slave-07c2 krb5kdc[4434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:59:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 00:59:05.505110 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:46641
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:46641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:05.824056 4448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:05.824378 4448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:05.824534 4448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:05.831418 4448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:05.831552 4448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:05.831662 4448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:05.831763 4448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:05.831849 4448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:05.841183 4448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:46641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:46641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:05.842460 4448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:05.843755 4448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:05.855649 4453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:05.855687 4454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:05.856213 4448 server_base.cc:1034] running on GCE node
W20260809 00:59:05.856598 4456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:05.857331 4448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:05.858644 4448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:05.859858 4448 hybrid_clock.cc:648] HybridClock initialized: now 1786237145859857 us; error 74 us; skew 500 ppm
I20260809 00:59:05.860180 4448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:05 dist-test-slave-07c2 krb5kdc[4434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237145, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:05.863096 4448 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 00:59:05.864547 4448 webserver.cc:460] Webserver started at http://127.4.82.190:45189/ using document root <none> and password file <none>
I20260809 00:59:05.865154 4448 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:05.865253 4448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:05.865499 4448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:05.868554 4448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "24cfeb3cc5604d22a1f7ccb4ca541052"
format_stamp: "Formatted at 2026-08-09 00:59:05 on dist-test-slave-07c2"
server_key: "8fa13ffbb9a731b4c5cd0ddf246cbb06"
server_key_iv: "5abdd60e2e7bd78bea4dd18b1b24708a"
server_key_version: "encryptionkey@0"
I20260809 00:59:05.869249 4448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "24cfeb3cc5604d22a1f7ccb4ca541052"
format_stamp: "Formatted at 2026-08-09 00:59:05 on dist-test-slave-07c2"
server_key: "8fa13ffbb9a731b4c5cd0ddf246cbb06"
server_key_iv: "5abdd60e2e7bd78bea4dd18b1b24708a"
server_key_version: "encryptionkey@0"
I20260809 00:59:05.874132 4448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 00:59:05.877374 4465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:05.878566 4448 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 00:59:05.878746 4448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "24cfeb3cc5604d22a1f7ccb4ca541052"
format_stamp: "Formatted at 2026-08-09 00:59:05 on dist-test-slave-07c2"
server_key: "8fa13ffbb9a731b4c5cd0ddf246cbb06"
server_key_iv: "5abdd60e2e7bd78bea4dd18b1b24708a"
server_key_version: "encryptionkey@0"
I20260809 00:59:05.878932 4448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:05.896883 4448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:05.900547 4448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:05.900818 4448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:05.919471 4448 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:46641
I20260809 00:59:05.919514 4518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:46641 every 8 connection(s)
I20260809 00:59:05.920768 4448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:05.924237 4519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:05.930137 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 4448
I20260809 00:59:05.930450 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:05.932981 4426 external_mini_cluster.cc:1428] Setting key a58b15d1938d1b9eefe727f50e46912c
I20260809 00:59:05.936914 4519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052: Bootstrap starting.
I20260809 00:59:05.940109 4519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:05.941072 4519 log.cc:824] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:05.943905 4519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052: No bootstrap required, opened a new log
I20260809 00:59:05.953610 4519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cfeb3cc5604d22a1f7ccb4ca541052" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46641 } }
I20260809 00:59:05.954015 4519 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:05.954113 4519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24cfeb3cc5604d22a1f7ccb4ca541052, State: Initialized, Role: FOLLOWER
I20260809 00:59:05.954666 4519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cfeb3cc5604d22a1f7ccb4ca541052" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46641 } }
I20260809 00:59:05.955061 4519 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:05.955274 4519 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:05.955585 4519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:05.958365 4519 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cfeb3cc5604d22a1f7ccb4ca541052" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46641 } }
I20260809 00:59:05.958889 4519 leader_election.cc:302] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24cfeb3cc5604d22a1f7ccb4ca541052; no voters:
I20260809 00:59:05.959669 4519 leader_election.cc:288] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:05.959757 4524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:05.960949 4524 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [term 1 LEADER]: Becoming Leader. State: Replica: 24cfeb3cc5604d22a1f7ccb4ca541052, State: Running, Role: LEADER
I20260809 00:59:05.961494 4524 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cfeb3cc5604d22a1f7ccb4ca541052" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46641 } }
I20260809 00:59:05.961936 4519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 00:59:05 dist-test-slave-07c2 krb5kdc[4434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:05.964450 4525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24cfeb3cc5604d22a1f7ccb4ca541052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cfeb3cc5604d22a1f7ccb4ca541052" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46641 } } }
I20260809 00:59:05.964890 4525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:05.967224 4527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24cfeb3cc5604d22a1f7ccb4ca541052. Latest consensus state: current_term: 1 leader_uuid: "24cfeb3cc5604d22a1f7ccb4ca541052" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24cfeb3cc5604d22a1f7ccb4ca541052" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46641 } } }
I20260809 00:59:05.967600 4527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:05.969378 4530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:05.974505 4530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:05.977671 4521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:05.937577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46150 (local address 127.4.82.190:46641)
0809 00:59:05.938526 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:05.938541 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:05.939832 (+ 1291us) server_negotiation.cc:407] Connection header received
0809 00:59:05.948433 (+ 8601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:05.948455 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:05.948937 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:05.949351 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:05.952297 (+ 2946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:05.954228 (+ 1931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:05.956277 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:05.957340 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:05.957530 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:05.964066 (+ 6536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:05.964081 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:05.964099 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:05.964155 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:05.969178 (+ 5023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:05.970352 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:05.970367 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:05.970376 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:05.970646 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:05.971362 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:05.971369 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:05.971376 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:05.972068 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:05.975098 (+ 3030us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:05.975472 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 00:59:05.975965 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":441,"threads_started":1}
I20260809 00:59:05.987869 4530 catalog_manager.cc:1347] Generated new cluster ID: 886693222b2c445581204b66c2c7b7d8
I20260809 00:59:05.987979 4530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:06.017693 4530 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:06.019167 4530 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:06.032998 4530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24cfeb3cc5604d22a1f7ccb4ca541052: Generated new TSK 0
I20260809 00:59:06.033737 4530 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 00:59:06.086570 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46641
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:06.422438 4549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:06.422701 4549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:06.422863 4549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:06.429658 4549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:06.429783 4549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:06.429947 4549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 00:59:06.439031 4549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46641
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:06.440153 4549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:06.441350 4549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:06.453828 4556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:06.453866 4555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:06.453887 4558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:06.455137 4549 server_base.cc:1034] running on GCE node
I20260809 00:59:06.455704 4549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:06.456460 4549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:06.457716 4549 hybrid_clock.cc:648] HybridClock initialized: now 1786237146457662 us; error 46 us; skew 500 ppm
I20260809 00:59:06.458077 4549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:06 dist-test-slave-07c2 krb5kdc[4434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237146, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:06.461192 4549 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 00:59:06.462616 4549 webserver.cc:460] Webserver started at http://127.4.82.129:33053/ using document root <none> and password file <none>
I20260809 00:59:06.463382 4549 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:06.463521 4549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:06.463881 4549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:06.466621 4549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "746d459a113a4f458501cd987d0b2f43"
format_stamp: "Formatted at 2026-08-09 00:59:06 on dist-test-slave-07c2"
server_key: "6f4d661604099a4447e4c67a06ef5fd1"
server_key_iv: "13bde1f6ea06e5490ce2d88e08e83505"
server_key_version: "encryptionkey@0"
I20260809 00:59:06.467504 4549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "746d459a113a4f458501cd987d0b2f43"
format_stamp: "Formatted at 2026-08-09 00:59:06 on dist-test-slave-07c2"
server_key: "6f4d661604099a4447e4c67a06ef5fd1"
server_key_iv: "13bde1f6ea06e5490ce2d88e08e83505"
server_key_version: "encryptionkey@0"
I20260809 00:59:06.472887 4549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 00:59:06.476421 4566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:06.477455 4549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 00:59:06.477640 4549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "746d459a113a4f458501cd987d0b2f43"
format_stamp: "Formatted at 2026-08-09 00:59:06 on dist-test-slave-07c2"
server_key: "6f4d661604099a4447e4c67a06ef5fd1"
server_key_iv: "13bde1f6ea06e5490ce2d88e08e83505"
server_key_version: "encryptionkey@0"
I20260809 00:59:06.477816 4549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:06.512207 4549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:06.515818 4549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:06.516089 4549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:06.516990 4549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:06.518263 4549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:06.518374 4549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:06.518515 4549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:06.518592 4549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:06.551683 4549 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:37619
I20260809 00:59:06.551764 4682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:37619 every 8 connection(s)
I20260809 00:59:06.552960 4549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 00:59:06.553728 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 4549
I20260809 00:59:06.553951 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 00:59:06.554858 4426 external_mini_cluster.cc:1428] Setting key 45674c3c2e23b06e6dceec502cc575fb
Aug 09 00:59:06 dist-test-slave-07c2 krb5kdc[4434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237146, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
I20260809 00:59:06.574064 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:06.556206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:48421 (local address 127.4.82.190:46641)
0809 00:59:06.556698 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:06.556704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:06.557788 (+ 1084us) server_negotiation.cc:407] Connection header received
0809 00:59:06.559649 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:06.559654 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:06.559744 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:06.559854 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:06.561285 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:06.562002 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:06.563608 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:06.563994 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:06.564144 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:06.568130 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:06.568134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:06.568138 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:06.568167 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:06.570800 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:06.571588 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:06.571594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:06.571597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:06.571726 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:06.572257 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:06.572262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:06.572265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:06.572526 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:06.572686 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:06.573474 (+ 788us) server_negotiation.cc:299] Negotiation successful
0809 00:59:06.573691 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":249,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
I20260809 00:59:06.575096 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:06.556874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:46641 (local address 127.4.82.129:48421)
0809 00:59:06.557632 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:06.557666 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:06.559405 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:06.560083 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:06.560091 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:06.560600 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:06.561124 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:06.561147 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:06.562168 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:06.562173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:06.563476 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:06.563488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:06.564227 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:06.564236 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:06.564458 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:06.565168 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:06.565191 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:06.567919 (+ 2728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:06.570916 (+ 2997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:06.570924 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:06.570938 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:06.571473 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:06.571856 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:06.571860 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:06.571864 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:06.572129 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:06.572661 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:06.572668 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:06.573057 (+ 389us) client_negotiation.cc:769] Sending connection context
0809 00:59:06.573371 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 00:59:06.573867 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":332,"threads_started":1}
I20260809 00:59:06.576675 4683 heartbeater.cc:347] Connected to a master server at 127.4.82.190:46641
I20260809 00:59:06.576979 4683 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:06.577665 4683 heartbeater.cc:511] Master 127.4.82.190:46641 requested a full tablet report, sending...
I20260809 00:59:06.579743 4482 ts_manager.cc:194] Registered new tserver with Master: 746d459a113a4f458501cd987d0b2f43 (127.4.82.129:37619)
I20260809 00:59:06.581805 4482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:48421
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 00:59:06.596374 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46641
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:06.907025 4689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:06.907265 4689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:06.907423 4689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:06.914367 4689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:06.914460 4689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:06.914579 4689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 00:59:06.923759 4689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46641
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:06.924839 4689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:06.926083 4689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:06.938470 4694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:06.938511 4695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:06.938697 4697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:06.939560 4689 server_base.cc:1034] running on GCE node
I20260809 00:59:06.940114 4689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:06.940920 4689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:06.942126 4689 hybrid_clock.cc:648] HybridClock initialized: now 1786237146942090 us; error 46 us; skew 500 ppm
I20260809 00:59:06.942409 4689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:06 dist-test-slave-07c2 krb5kdc[4434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237146, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:06.945346 4689 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 00:59:06.946662 4689 webserver.cc:460] Webserver started at http://127.4.82.130:35747/ using document root <none> and password file <none>
I20260809 00:59:06.947266 4689 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:06.947451 4689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:06.947751 4689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:06.950469 4689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "0f76f27b2bfc487998aaad0b7a85a5ac"
format_stamp: "Formatted at 2026-08-09 00:59:06 on dist-test-slave-07c2"
server_key: "245a6040984e07d2e58cae8b3cadf3fb"
server_key_iv: "0ac12196b94566228cf89006efed1f8c"
server_key_version: "encryptionkey@0"
I20260809 00:59:06.951198 4689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "0f76f27b2bfc487998aaad0b7a85a5ac"
format_stamp: "Formatted at 2026-08-09 00:59:06 on dist-test-slave-07c2"
server_key: "245a6040984e07d2e58cae8b3cadf3fb"
server_key_iv: "0ac12196b94566228cf89006efed1f8c"
server_key_version: "encryptionkey@0"
I20260809 00:59:06.955943 4689 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 00:59:06.959136 4705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:06.960441 4689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 00:59:06.960644 4689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "0f76f27b2bfc487998aaad0b7a85a5ac"
format_stamp: "Formatted at 2026-08-09 00:59:06 on dist-test-slave-07c2"
server_key: "245a6040984e07d2e58cae8b3cadf3fb"
server_key_iv: "0ac12196b94566228cf89006efed1f8c"
server_key_version: "encryptionkey@0"
I20260809 00:59:06.960834 4689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:06.993458 4689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:06.997175 4689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:06.997455 4689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:06.998366 4689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:06.999729 4689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:06.999827 4689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:06.999969 4689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:07.000039 4689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:07.033635 4689 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:38253
I20260809 00:59:07.033676 4827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:38253 every 8 connection(s)
I20260809 00:59:07.034897 4689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 00:59:07.040194 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 4689
I20260809 00:59:07.040401 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 00:59:07.040964 4426 external_mini_cluster.cc:1428] Setting key 0e704a6ab2642df8cfa684a11687d9d1
Aug 09 00:59:07 dist-test-slave-07c2 krb5kdc[4434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237146, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:07.055047 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:07.037393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:37745 (local address 127.4.82.190:46641)
0809 00:59:07.037607 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:07.037613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:07.038753 (+ 1140us) server_negotiation.cc:407] Connection header received
0809 00:59:07.040622 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:07.040628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:07.040715 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:07.040839 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:07.042562 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.043426 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.044987 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.045776 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.045923 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:07.049352 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:07.049357 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:07.049360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:07.049391 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:07.052078 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.052787 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.052792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.052796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.052920 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.053350 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.053356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.053359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.053632 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:07.053794 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:07.054478 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 00:59:07.054699 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 00:59:07.056197 4831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:07.037891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:46641 (local address 127.4.82.130:37745)
0809 00:59:07.038592 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:07.038637 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:07.040295 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:07.041017 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:07.041024 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:07.041653 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:07.042407 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:07.042430 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.043575 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.043579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:07.044844 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:07.044856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.045983 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.045988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:07.046173 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:07.046854 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:07.046882 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:07.049135 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:07.052223 (+ 3088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:07.052232 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:07.052249 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:07.052657 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:07.053039 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:07.053045 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:07.053048 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:07.053240 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:07.053765 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:07.053772 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:07.054103 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 00:59:07.054383 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 00:59:07.054889 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":297,"threads_started":1}
I20260809 00:59:07.057667 4828 heartbeater.cc:347] Connected to a master server at 127.4.82.190:46641
I20260809 00:59:07.057974 4828 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
I20260809 00:59:07.058699 4828 heartbeater.cc:511] Master 127.4.82.190:46641 requested a full tablet report, sending...
I20260809 00:59:07.060111 4482 ts_manager.cc:194] Registered new tserver with Master: 0f76f27b2bfc487998aaad0b7a85a5ac (127.4.82.130:38253)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
I20260809 00:59:07.060990 4482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:37745
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 00:59:07.082798 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46641
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:07.408727 4834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:07.408985 4834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:07.409081 4834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:07.416046 4834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:07.416145 4834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:07.416255 4834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 00:59:07.425231 4834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46641
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:07.426311 4834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:07.427526 4834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:07.440363 4840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:07.440372 4842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:07.440248 4839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:07.441123 4834 server_base.cc:1034] running on GCE node
I20260809 00:59:07.441633 4834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:07.442421 4834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:07.443598 4834 hybrid_clock.cc:648] HybridClock initialized: now 1786237147443541 us; error 49 us; skew 500 ppm
I20260809 00:59:07.443902 4834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:07 dist-test-slave-07c2 krb5kdc[4434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237147, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:07.446954 4834 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 00:59:07.448477 4834 webserver.cc:460] Webserver started at http://127.4.82.131:38593/ using document root <none> and password file <none>
I20260809 00:59:07.449111 4834 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:07.449254 4834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:07.449555 4834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:07.452684 4834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "39adc79639e7409c83571c8a085afe17"
format_stamp: "Formatted at 2026-08-09 00:59:07 on dist-test-slave-07c2"
server_key: "2676221e3d0fd46d3c40531e60d1ebc2"
server_key_iv: "362c1767ee328ff66897c7349128cda1"
server_key_version: "encryptionkey@0"
I20260809 00:59:07.453438 4834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "39adc79639e7409c83571c8a085afe17"
format_stamp: "Formatted at 2026-08-09 00:59:07 on dist-test-slave-07c2"
server_key: "2676221e3d0fd46d3c40531e60d1ebc2"
server_key_iv: "362c1767ee328ff66897c7349128cda1"
server_key_version: "encryptionkey@0"
I20260809 00:59:07.458468 4834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 00:59:07.461764 4850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:07.463018 4834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 00:59:07.463188 4834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "39adc79639e7409c83571c8a085afe17"
format_stamp: "Formatted at 2026-08-09 00:59:07 on dist-test-slave-07c2"
server_key: "2676221e3d0fd46d3c40531e60d1ebc2"
server_key_iv: "362c1767ee328ff66897c7349128cda1"
server_key_version: "encryptionkey@0"
I20260809 00:59:07.463430 4834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:07.480492 4834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:07.484311 4834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:07.484583 4834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:07.485469 4834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:07.486732 4834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:07.486831 4834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:07.486979 4834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:07.487051 4834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:07.520629 4834 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:38529
I20260809 00:59:07.520678 4965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:38529 every 8 connection(s)
I20260809 00:59:07.521919 4834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 00:59:07.529474 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 4834
I20260809 00:59:07.529693 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 00:59:07.530149 4426 external_mini_cluster.cc:1428] Setting key 0c5c08341725fe47166a79344afbc1e8
Aug 09 00:59:07 dist-test-slave-07c2 krb5kdc[4434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237147, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:07.542133 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:07.524489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:34101 (local address 127.4.82.190:46641)
0809 00:59:07.524663 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:07.524668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:07.526004 (+ 1336us) server_negotiation.cc:407] Connection header received
0809 00:59:07.527871 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:07.527877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:07.527963 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:07.528078 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:07.529826 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.530636 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.532900 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.533528 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.533693 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:07.537161 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:07.537167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:07.537170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:07.537204 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:07.539621 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.540293 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.540297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.540299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.540432 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.540780 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.540783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.540785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.541033 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:07.541141 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:07.541745 (+ 604us) server_negotiation.cc:299] Negotiation successful
0809 00:59:07.541895 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 00:59:07.542999 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:07.524912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:46641 (local address 127.4.82.131:34101)
0809 00:59:07.525622 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:07.525666 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:07.527619 (+ 1953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:07.528295 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:07.528303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:07.528838 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:07.529607 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:07.529635 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.530808 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.530815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:07.532770 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:07.532781 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.533789 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.533796 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:07.534052 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:07.534685 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:07.534741 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:07.536972 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:07.539755 (+ 2783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:07.539762 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:07.539772 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:07.540174 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:07.540520 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:07.540522 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:07.540525 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:07.540686 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:07.541130 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:07.541135 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:07.541430 (+ 295us) client_negotiation.cc:769] Sending connection context
0809 00:59:07.541647 (+ 217us) client_negotiation.cc:240] Negotiation successful
0809 00:59:07.541986 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":293,"threads_started":1}
I20260809 00:59:07.544404 4966 heartbeater.cc:347] Connected to a master server at 127.4.82.190:46641
I20260809 00:59:07.544691 4966 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:07.545373 4966 heartbeater.cc:511] Master 127.4.82.190:46641 requested a full tablet report, sending...
I20260809 00:59:07.546759 4482 ts_manager.cc:194] Registered new tserver with Master: 39adc79639e7409c83571c8a085afe17 (127.4.82.131:38529)
I20260809 00:59:07.547606 4482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:34101
I20260809 00:59:07.557974 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:59:07.584717 4683 heartbeater.cc:503] Master 127.4.82.190:46641 was elected leader, sending a full tablet report...
Aug 09 00:59:07 dist-test-slave-07c2 krb5kdc[4434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237147, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:59:07 dist-test-slave-07c2 krb5kdc[4434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237147, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 00:59:07.611608 4972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:07.597027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58188 (local address 127.4.82.129:37619)
0809 00:59:07.597627 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:07.597634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:07.597750 (+ 116us) server_negotiation.cc:407] Connection header received
0809 00:59:07.597889 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:07.597896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:07.598117 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:07.598247 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:07.599708 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.600697 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.601886 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.602404 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.602500 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:07.606300 (+ 3800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:07.606312 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:07.606326 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:07.606360 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:07.609262 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.609888 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.609894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.609896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.610013 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.610322 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.610325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.610340 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.610642 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:07.610896 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:07.611082 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 00:59:07.611266 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":314,"threads_started":1}
W20260809 00:59:07.612792 4595 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58188
Aug 09 00:59:07 dist-test-slave-07c2 krb5kdc[4434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:59:07 dist-test-slave-07c2 krb5kdc[4434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 00:59:07.659168 4972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:07.644209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58196 (local address 127.4.82.129:37619)
0809 00:59:07.644471 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:07.644478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:07.644876 (+ 398us) server_negotiation.cc:407] Connection header received
0809 00:59:07.645072 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:07.645076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:07.645156 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:07.645238 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:07.647120 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.648193 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.650116 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.650533 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.650663 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:07.654562 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:07.654567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:07.654571 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:07.654603 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:07.657176 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.657729 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.657733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.657735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.657852 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.658137 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.658140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.658142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.658375 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:07.658546 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:07.658783 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 00:59:07.658938 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 00:59:07.660640 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 4549
I20260809 00:59:07.673295 4972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:07.661656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58200 (local address 127.4.82.129:37619)
0809 00:59:07.661868 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:07.661874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:07.662030 (+ 156us) server_negotiation.cc:407] Connection header received
0809 00:59:07.662155 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:07.662167 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:07.662221 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:07.662289 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:07.663539 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.664197 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.665520 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.665922 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.666011 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:07.668387 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:07.668392 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:07.668395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:07.668423 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:07.670974 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.671749 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.671753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.671755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.671872 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.672208 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.672220 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.672223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.672471 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:07.672632 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:07.672908 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 00:59:07.673044 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 00:59:07.674028 4678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:07.756062 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 4549
I20260809 00:59:07.774348 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 4689
Aug 09 00:59:07 dist-test-slave-07c2 krb5kdc[4434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 00:59:07.789707 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:07.775337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36730 (local address 127.4.82.130:38253)
0809 00:59:07.775855 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:07.775862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:07.775887 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:59:07.775952 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:07.775958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:07.776150 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:07.776283 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:07.777408 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.778387 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.779640 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.780162 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.780279 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:07.784291 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:07.784301 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:07.784313 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:07.784342 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:07.787137 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.787878 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.787885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.787887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.788012 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.788343 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.788346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.788348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.788682 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:07.788919 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:07.789201 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 00:59:07.789389 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":265,"threads_started":1}
I20260809 00:59:07.790910 4823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:07.872501 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 4689
I20260809 00:59:07.889726 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 4834
Aug 09 00:59:07 dist-test-slave-07c2 krb5kdc[4434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 00:59:07.906388 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:07.890802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60062 (local address 127.4.82.131:38529)
0809 00:59:07.891498 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:07.891505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:07.891535 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:59:07.891598 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:07.891604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:07.891809 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:07.891942 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:07.893319 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.894563 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.895923 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:07.896454 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:07.896602 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:07.900712 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:07.900723 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:07.900737 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:07.900768 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:07.903639 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.904300 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.904307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.904309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.904434 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:07.904800 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:07.904806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:07.904809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:07.905149 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:07.905456 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:07.905779 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 00:59:07.906043 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":335,"threads_started":1}
I20260809 00:59:07.907716 4961 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:07.988817 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 4834
I20260809 00:59:08.006378 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 4448
I20260809 00:59:08.007674 4514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:08.111666 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 4448
2026-08-09T00:59:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3905 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:59:08 dist-test-slave-07c2 krb5kdc[4989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:59:08 dist-test-slave-07c2 krb5kdc[4989](info): set up 2 sockets
Aug 09 00:59:08 dist-test-slave-07c2 krb5kdc[4989](info): commencing operation
krb5kdc: starting...
W20260809 00:59:09.222048 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 00:59:09 dist-test-slave-07c2 krb5kdc[4989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:59:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 00:59:09.396564 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:37933
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:37933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:09.703917 5002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:09.704228 5002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:09.704391 5002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:09.711196 5002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:09.711367 5002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:09.711463 5002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:09.711558 5002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:09.711640 5002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:09.720989 5002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:37933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:37933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:09.722144 5002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:09.723342 5002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:09.734280 5007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:09.734323 5010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:09.734380 5008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:09.735251 5002 server_base.cc:1034] running on GCE node
I20260809 00:59:09.735945 5002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:09.737138 5002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:09.738363 5002 hybrid_clock.cc:648] HybridClock initialized: now 1786237149738321 us; error 46 us; skew 500 ppm
I20260809 00:59:09.738696 5002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:09 dist-test-slave-07c2 krb5kdc[4989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237149, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:09.745605 5002 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 00:59:09.746930 5002 webserver.cc:460] Webserver started at http://127.4.82.190:44849/ using document root <none> and password file <none>
I20260809 00:59:09.747651 5002 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:09.747794 5002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:09.748097 5002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:09.750999 5002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "366521111116489dab0989f9ce24b716"
format_stamp: "Formatted at 2026-08-09 00:59:09 on dist-test-slave-07c2"
server_key: "7446d30dd3e45d5ce6b9f1258a2c6821"
server_key_iv: "783d004e034e139a338654639bf59108"
server_key_version: "encryptionkey@0"
I20260809 00:59:09.751762 5002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "366521111116489dab0989f9ce24b716"
format_stamp: "Formatted at 2026-08-09 00:59:09 on dist-test-slave-07c2"
server_key: "7446d30dd3e45d5ce6b9f1258a2c6821"
server_key_iv: "783d004e034e139a338654639bf59108"
server_key_version: "encryptionkey@0"
I20260809 00:59:09.756705 5002 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 00:59:09.759871 5018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:09.761027 5002 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:59:09.761200 5002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "366521111116489dab0989f9ce24b716"
format_stamp: "Formatted at 2026-08-09 00:59:09 on dist-test-slave-07c2"
server_key: "7446d30dd3e45d5ce6b9f1258a2c6821"
server_key_iv: "783d004e034e139a338654639bf59108"
server_key_version: "encryptionkey@0"
I20260809 00:59:09.761407 5002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:09.790661 5002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:09.794780 5002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:09.795065 5002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:09.813181 5002 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:37933
I20260809 00:59:09.813221 5071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:37933 every 8 connection(s)
I20260809 00:59:09.814420 5002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:09.817811 5072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:09.820658 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 5002
I20260809 00:59:09.820899 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:09.821563 4426 external_mini_cluster.cc:1428] Setting key 5e6cf927f9ce7776cc93db0fa006420b
I20260809 00:59:09.830327 5072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716: Bootstrap starting.
I20260809 00:59:09.833554 5072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:09.834441 5072 log.cc:824] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716: Log is configured to *not* fsync() on all Append() calls
Aug 09 00:59:09 dist-test-slave-07c2 krb5kdc[4989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:09.837405 5072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716: No bootstrap required, opened a new log
I20260809 00:59:09.845954 5075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:09.823416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58246 (local address 127.4.82.190:37933)
0809 00:59:09.824261 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:09.824278 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:09.824328 (+ 50us) server_negotiation.cc:407] Connection header received
0809 00:59:09.826243 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:09.826271 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:09.826761 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:09.827165 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:09.828485 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:09.830380 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:09.831618 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:09.832652 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:09.832847 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:09.836865 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:09.836876 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:09.836892 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:09.836939 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:09.841341 (+ 4402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:09.842005 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:09.842014 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:09.842022 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:09.842290 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:09.842629 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:09.842634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:09.842638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:09.843360 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:09.843705 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:09.844046 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 00:59:09.844511 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":360,"threads_started":1}
I20260809 00:59:09.849210 5072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366521111116489dab0989f9ce24b716" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37933 } }
I20260809 00:59:09.849534 5072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:09.849623 5072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 366521111116489dab0989f9ce24b716, State: Initialized, Role: FOLLOWER
I20260809 00:59:09.850109 5072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366521111116489dab0989f9ce24b716" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37933 } }
I20260809 00:59:09.850325 5072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:09.850425 5072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:09.850540 5072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:09.852735 5072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366521111116489dab0989f9ce24b716" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37933 } }
I20260809 00:59:09.853168 5072 leader_election.cc:302] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 366521111116489dab0989f9ce24b716; no voters:
I20260809 00:59:09.853678 5072 leader_election.cc:288] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:09.853802 5077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:09.855218 5077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [term 1 LEADER]: Becoming Leader. State: Replica: 366521111116489dab0989f9ce24b716, State: Running, Role: LEADER
I20260809 00:59:09.855747 5077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366521111116489dab0989f9ce24b716" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37933 } }
I20260809 00:59:09.855878 5072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:09.858757 5079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 366521111116489dab0989f9ce24b716. Latest consensus state: current_term: 1 leader_uuid: "366521111116489dab0989f9ce24b716" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366521111116489dab0989f9ce24b716" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37933 } } }
I20260809 00:59:09.859275 5079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:09.860050 5078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "366521111116489dab0989f9ce24b716" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366521111116489dab0989f9ce24b716" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37933 } } }
I20260809 00:59:09.860424 5078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:09.863878 5084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:09.869269 5084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:09.877564 5084 catalog_manager.cc:1347] Generated new cluster ID: 444ab19bf2c54f53bf1505fa061523c0
I20260809 00:59:09.877689 5084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:09.891897 5084 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:09.893069 5084 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:09.907424 5084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 366521111116489dab0989f9ce24b716: Generated new TSK 0
I20260809 00:59:09.908313 5084 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 00:59:09.952035 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37933
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:10.267135 5098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:10.267417 5098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:10.267580 5098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:10.274559 5098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:10.274663 5098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:10.274787 5098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 00:59:10.285032 5098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37933
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:10.286195 5098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:10.287496 5098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:10.300978 5106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:10.301085 5104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:10.301070 5103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:10.301952 5098 server_base.cc:1034] running on GCE node
I20260809 00:59:10.302508 5098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:10.303428 5098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:10.304638 5098 hybrid_clock.cc:648] HybridClock initialized: now 1786237150304600 us; error 28 us; skew 500 ppm
I20260809 00:59:10.304939 5098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:10 dist-test-slave-07c2 krb5kdc[4989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237150, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:10.307909 5098 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 00:59:10.309291 5098 webserver.cc:460] Webserver started at http://127.4.82.129:44853/ using document root <none> and password file <none>
I20260809 00:59:10.309922 5098 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:10.310050 5098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:10.310357 5098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:10.313098 5098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c"
format_stamp: "Formatted at 2026-08-09 00:59:10 on dist-test-slave-07c2"
server_key: "2fe5f06c7e3727ba7a4196da69a64598"
server_key_iv: "2cb60b6f1e69474739b90cc2c91c1890"
server_key_version: "encryptionkey@0"
I20260809 00:59:10.313872 5098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c"
format_stamp: "Formatted at 2026-08-09 00:59:10 on dist-test-slave-07c2"
server_key: "2fe5f06c7e3727ba7a4196da69a64598"
server_key_iv: "2cb60b6f1e69474739b90cc2c91c1890"
server_key_version: "encryptionkey@0"
I20260809 00:59:10.318699 5098 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 00:59:10.321955 5113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:10.323105 5098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 00:59:10.323252 5098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c"
format_stamp: "Formatted at 2026-08-09 00:59:10 on dist-test-slave-07c2"
server_key: "2fe5f06c7e3727ba7a4196da69a64598"
server_key_iv: "2cb60b6f1e69474739b90cc2c91c1890"
server_key_version: "encryptionkey@0"
I20260809 00:59:10.323483 5098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:10.349152 5098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:10.352854 5098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:10.353116 5098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:10.354064 5098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:10.355357 5098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:10.355436 5098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:10.355592 5098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:10.355654 5098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:10.387923 5098 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:42423
I20260809 00:59:10.388000 5230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:42423 every 8 connection(s)
I20260809 00:59:10.389129 5098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 00:59:10.399380 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 5098
I20260809 00:59:10.399618 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 00:59:10.400445 4426 external_mini_cluster.cc:1428] Setting key 05cfda46541d0d90506bbcf0438c6fb2
Aug 09 00:59:10 dist-test-slave-07c2 krb5kdc[4989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237150, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:10.409054 5233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:10.391486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:40833 (local address 127.4.82.190:37933)
0809 00:59:10.391937 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:10.391944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:10.392848 (+ 904us) server_negotiation.cc:407] Connection header received
0809 00:59:10.394505 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:10.394509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:10.394585 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:10.394694 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:10.396059 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:10.396843 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:10.398361 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:10.398765 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:10.398869 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:10.403006 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:10.403011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:10.403014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:10.403046 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:10.405584 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:10.406527 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:10.406532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:10.406535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:10.406733 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:10.407216 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:10.407219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:10.407221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:10.407569 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:10.407776 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:10.408566 (+ 790us) server_negotiation.cc:299] Negotiation successful
0809 00:59:10.408735 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":202,"threads_started":1}
I20260809 00:59:10.410195 5234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:10.392038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:37933 (local address 127.4.82.129:40833)
0809 00:59:10.392697 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:10.392738 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:10.394257 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:10.394856 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:10.394863 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:10.395385 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:10.395893 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:10.395928 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:10.396963 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:10.396967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:10.398236 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:10.398252 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:10.399007 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:10.399017 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:10.399367 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:10.400088 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:10.400118 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:10.402809 (+ 2691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:10.405756 (+ 2947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:10.405767 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:10.405783 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:10.406379 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:10.406843 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:10.406851 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:10.406857 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:10.407090 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:10.407750 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:10.407760 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:10.408165 (+ 405us) client_negotiation.cc:769] Sending connection context
0809 00:59:10.408482 (+ 317us) client_negotiation.cc:240] Negotiation successful
0809 00:59:10.409009 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":273,"threads_started":1}
I20260809 00:59:10.411651 5231 heartbeater.cc:347] Connected to a master server at 127.4.82.190:37933
I20260809 00:59:10.411983 5231 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:10.412799 5231 heartbeater.cc:511] Master 127.4.82.190:37933 requested a full tablet report, sending...
I20260809 00:59:10.414844 5035 ts_manager.cc:194] Registered new tserver with Master: 87eb9a9c7c6b49f29f0a5fc550f4635c (127.4.82.129:42423)
I20260809 00:59:10.416304 5035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:40833
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 00:59:10.443787 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37933
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:10.759660 5237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:10.759922 5237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:10.760106 5237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:10.767378 5237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:10.767489 5237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:10.767663 5237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 00:59:10.777128 5237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37933
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:10.778301 5237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:10.779563 5237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:10.792011 5243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:10.791963 5246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:10.792264 5244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:10.792937 5237 server_base.cc:1034] running on GCE node
I20260809 00:59:10.793514 5237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:10.794334 5237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:10.795522 5237 hybrid_clock.cc:648] HybridClock initialized: now 1786237150795461 us; error 48 us; skew 500 ppm
I20260809 00:59:10.795833 5237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:10 dist-test-slave-07c2 krb5kdc[4989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237150, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:10.799239 5237 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 00:59:10.800635 5237 webserver.cc:460] Webserver started at http://127.4.82.130:35587/ using document root <none> and password file <none>
I20260809 00:59:10.801260 5237 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:10.801355 5237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:10.801597 5237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:10.804430 5237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "ef3f8c7b22164707856193e75a6729bb"
format_stamp: "Formatted at 2026-08-09 00:59:10 on dist-test-slave-07c2"
server_key: "ffd48b38984c0534ac88cf79fbf790a2"
server_key_iv: "235073395a814755ba38f1cd87790b3a"
server_key_version: "encryptionkey@0"
I20260809 00:59:10.805177 5237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "ef3f8c7b22164707856193e75a6729bb"
format_stamp: "Formatted at 2026-08-09 00:59:10 on dist-test-slave-07c2"
server_key: "ffd48b38984c0534ac88cf79fbf790a2"
server_key_iv: "235073395a814755ba38f1cd87790b3a"
server_key_version: "encryptionkey@0"
I20260809 00:59:10.810011 5237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 00:59:10.813333 5253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:10.814836 5237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 00:59:10.814998 5237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "ef3f8c7b22164707856193e75a6729bb"
format_stamp: "Formatted at 2026-08-09 00:59:10 on dist-test-slave-07c2"
server_key: "ffd48b38984c0534ac88cf79fbf790a2"
server_key_iv: "235073395a814755ba38f1cd87790b3a"
server_key_version: "encryptionkey@0"
I20260809 00:59:10.815227 5237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:10.836113 5237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:10.839704 5237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:10.839933 5237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:10.840821 5237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:10.842011 5237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:10.842092 5237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:10.842170 5237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:10.842216 5237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:10.873654 5237 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:35839
I20260809 00:59:10.873687 5373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:35839 every 8 connection(s)
I20260809 00:59:10.874869 5237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 00:59:10.880496 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 5237
I20260809 00:59:10.880712 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 00:59:10.881258 4426 external_mini_cluster.cc:1428] Setting key d5fea112b2662f1e86a2e553d1ddba88
Aug 09 00:59:10 dist-test-slave-07c2 krb5kdc[4989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237150, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:10.895839 5233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:10.877398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:37935 (local address 127.4.82.190:37933)
0809 00:59:10.877599 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:10.877606 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:10.878744 (+ 1138us) server_negotiation.cc:407] Connection header received
0809 00:59:10.880779 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:10.880789 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:10.880882 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:10.880994 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:10.882654 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:10.883676 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:10.885217 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:10.886019 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:10.886159 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:10.889804 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:10.889809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:10.889812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:10.889843 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:10.892537 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:10.893408 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:10.893412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:10.893414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:10.893532 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:10.894079 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:10.894082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:10.894084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:10.894313 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:10.894431 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:10.895234 (+ 803us) server_negotiation.cc:299] Negotiation successful
0809 00:59:10.895510 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 00:59:10.897200 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:10.877853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:37933 (local address 127.4.82.130:37935)
0809 00:59:10.878587 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:10.878623 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:10.880436 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:10.881175 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:10.881182 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:10.881731 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:10.882486 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:10.882510 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:10.883823 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:10.883827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:10.885052 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:10.885063 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:10.886243 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:10.886248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:10.886447 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:10.887102 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:10.887122 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:10.889569 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:10.892681 (+ 3112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:10.892692 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:10.892709 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:10.893306 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:10.893652 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:10.893660 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:10.893666 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:10.893963 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:10.894434 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:10.894444 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:10.894865 (+ 421us) client_negotiation.cc:769] Sending connection context
0809 00:59:10.895158 (+ 293us) client_negotiation.cc:240] Negotiation successful
0809 00:59:10.895740 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":304,"threads_started":1}
I20260809 00:59:10.898684 5374 heartbeater.cc:347] Connected to a master server at 127.4.82.190:37933
I20260809 00:59:10.899048 5374 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:10.899804 5374 heartbeater.cc:511] Master 127.4.82.190:37933 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
I20260809 00:59:10.901474 5035 ts_manager.cc:194] Registered new tserver with Master: ef3f8c7b22164707856193e75a6729bb (127.4.82.130:35839)
I20260809 00:59:10.902264 5035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:37935
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 00:59:10.924198 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37933
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:11.242739 5379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:11.243059 5379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:11.243230 5379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:11.250496 5379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:11.250658 5379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:11.250840 5379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 00:59:11.260380 5379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37933
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:11.261548 5379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:11.262804 5379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:11.275727 5388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:11.275835 5385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:11.275872 5384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:11.276582 5379 server_base.cc:1034] running on GCE node
I20260809 00:59:11.277194 5379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:11.278101 5379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:11.279371 5379 hybrid_clock.cc:648] HybridClock initialized: now 1786237151279319 us; error 55 us; skew 500 ppm
I20260809 00:59:11.279672 5379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:11 dist-test-slave-07c2 krb5kdc[4989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237151, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:11.283757 5379 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 00:59:11.285105 5379 webserver.cc:460] Webserver started at http://127.4.82.131:34193/ using document root <none> and password file <none>
I20260809 00:59:11.285792 5379 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:11.285935 5379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:11.286255 5379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:11.289155 5379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "05f19ede507d4456846711db330febd2"
format_stamp: "Formatted at 2026-08-09 00:59:11 on dist-test-slave-07c2"
server_key: "8eb9a1405cb9f939556981c6ec18aeca"
server_key_iv: "72e4d7f28370cacc17af396db2a91890"
server_key_version: "encryptionkey@0"
I20260809 00:59:11.289948 5379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "05f19ede507d4456846711db330febd2"
format_stamp: "Formatted at 2026-08-09 00:59:11 on dist-test-slave-07c2"
server_key: "8eb9a1405cb9f939556981c6ec18aeca"
server_key_iv: "72e4d7f28370cacc17af396db2a91890"
server_key_version: "encryptionkey@0"
I20260809 00:59:11.294842 5379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 00:59:11.298244 5396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:11.299633 5379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 00:59:11.299840 5379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "05f19ede507d4456846711db330febd2"
format_stamp: "Formatted at 2026-08-09 00:59:11 on dist-test-slave-07c2"
server_key: "8eb9a1405cb9f939556981c6ec18aeca"
server_key_iv: "72e4d7f28370cacc17af396db2a91890"
server_key_version: "encryptionkey@0"
I20260809 00:59:11.300045 5379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:11.311072 5379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:11.315447 5379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:11.315757 5379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:11.316642 5379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:11.317893 5379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:11.317996 5379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:11.318141 5379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:11.318217 5379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:11.351377 5379 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:44813
I20260809 00:59:11.351423 5516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:44813 every 8 connection(s)
I20260809 00:59:11.352612 5379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 00:59:11.359572 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 5379
I20260809 00:59:11.359766 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 00:59:11.360334 4426 external_mini_cluster.cc:1428] Setting key a4938b6a7693d3137f43abecc63284e0
Aug 09 00:59:11 dist-test-slave-07c2 krb5kdc[4989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237151, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:11.371719 5233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.355034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:34821 (local address 127.4.82.190:37933)
0809 00:59:11.355217 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:11.355222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:11.356493 (+ 1271us) server_negotiation.cc:407] Connection header received
0809 00:59:11.358256 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:11.358261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:11.358327 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:11.358421 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:11.360002 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.361066 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.362553 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.362978 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.363112 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.366516 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:11.366521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:11.366524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:11.366554 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:11.369245 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.369847 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.369851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.369854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.369970 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.370326 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.370328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.370330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.370568 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:11.370678 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:11.371253 (+ 575us) server_negotiation.cc:299] Negotiation successful
0809 00:59:11.371481 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 00:59:11.372581 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.355534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:37933 (local address 127.4.82.131:34821)
0809 00:59:11.356340 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:11.356385 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:11.358030 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:11.358614 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:11.358621 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:11.359109 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:11.359759 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:11.359779 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.361221 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.361225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:11.362407 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:11.362419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.363197 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.363206 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:11.363465 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.364046 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:11.364066 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:11.366347 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:11.369356 (+ 3009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:11.369361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:11.369371 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:11.369745 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:11.370057 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:11.370060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:11.370063 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:11.370227 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:11.370667 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:11.370672 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:11.370962 (+ 290us) client_negotiation.cc:769] Sending connection context
0809 00:59:11.371158 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 00:59:11.371566 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":352,"threads_started":1}
I20260809 00:59:11.373912 5517 heartbeater.cc:347] Connected to a master server at 127.4.82.190:37933
I20260809 00:59:11.374208 5517 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:11.374869 5517 heartbeater.cc:511] Master 127.4.82.190:37933 requested a full tablet report, sending...
I20260809 00:59:11.376282 5035 ts_manager.cc:194] Registered new tserver with Master: 05f19ede507d4456846711db330febd2 (127.4.82.131:44813)
I20260809 00:59:11.377075 5035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:34821
I20260809 00:59:11.387188 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:59:11.405812 5233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.393613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58256 (local address 127.4.82.190:37933)
0809 00:59:11.393804 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:11.393808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:11.394322 (+ 514us) server_negotiation.cc:407] Connection header received
0809 00:59:11.394452 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:11.394455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:11.394523 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:11.394604 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:11.396076 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.396766 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.397977 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.398367 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.398454 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.400762 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:11.400767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:11.400770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:11.400799 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:11.403478 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.404124 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.404129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.404131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.404275 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.404656 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.404661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.404663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.404952 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:11.405127 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:11.405443 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 00:59:11.405592 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 00:59:11.412076 5035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58256:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:59:11.418761 5035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:59:11.419647 5231 heartbeater.cc:503] Master 127.4.82.190:37933 was elected leader, sending a full tablet report...
I20260809 00:59:11.442664 5531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.431144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43202 (local address 127.4.82.129:42423)
0809 00:59:11.434496 (+ 3352us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:11.434507 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:11.434536 (+ 29us) server_negotiation.cc:407] Connection header received
0809 00:59:11.434632 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:11.434639 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:11.434901 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:11.435135 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:11.435926 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.437185 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.440558 (+ 3373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.441479 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.441634 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.441875 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:11.442147 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 00:59:11.442308 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3176,"thread_start_us":3049,"threads_started":1}
I20260809 00:59:11.442893 5529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.430976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:42423 (local address 127.0.0.1:43202)
0809 00:59:11.431832 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:11.431874 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:11.432063 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:11.435152 (+ 3089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:11.435158 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:11.435183 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:11.435727 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:11.435741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.437332 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.437340 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:11.440380 (+ 3040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:11.440392 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.441695 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.441702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:11.441908 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.441962 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 00:59:11.442096 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 00:59:11.442332 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":496,"thread_start_us":455,"threads_started":1}
I20260809 00:59:11.442929 5530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.431019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:44813 (local address 127.0.0.1:56130)
0809 00:59:11.432641 (+ 1622us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:11.432666 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:11.432776 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:11.435419 (+ 2643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:11.435427 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:11.435452 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:11.436347 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:11.436360 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.438049 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.438056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:11.439887 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:11.439898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.441814 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.441821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:11.441989 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.442036 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 00:59:11.442106 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:59:11.442317 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1481,"mutex_wait_us":464,"spinlock_wait_cycles":3968,"thread_start_us":1359,"threads_started":1}
I20260809 00:59:11.442960 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.432738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:35839 (local address 127.0.0.1:48370)
0809 00:59:11.433329 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:11.433353 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:11.433485 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:11.434820 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:11.434826 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:11.434852 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:11.435381 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:11.435396 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.437585 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.437590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:11.440072 (+ 2482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:11.440087 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.442086 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.442094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:11.442284 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.442314 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 00:59:11.442395 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 00:59:11.442534 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":80,"thread_start_us":778,"threads_started":1}
I20260809 00:59:11.443610 5534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.433307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48370 (local address 127.4.82.130:35839)
0809 00:59:11.434093 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:11.434106 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:11.434139 (+ 33us) server_negotiation.cc:407] Connection header received
0809 00:59:11.434261 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:11.434272 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:11.434545 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:11.434811 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:11.435601 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.437452 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.440239 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.441598 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.441736 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.441923 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:11.443056 (+ 1133us) server_negotiation.cc:299] Negotiation successful
0809 00:59:11.443246 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":411,"threads_started":1}
I20260809 00:59:11.444546 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.431769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56130 (local address 127.4.82.131:44813)
0809 00:59:11.434830 (+ 3061us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:11.434838 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:11.434867 (+ 29us) server_negotiation.cc:407] Connection header received
0809 00:59:11.434982 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:11.434993 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:11.435194 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:11.435427 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:11.436547 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.437920 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.440037 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.441548 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.443468 (+ 1920us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.443699 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:11.444104 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 00:59:11.444262 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2914,"thread_start_us":2754,"threads_started":1}
I20260809 00:59:11.446825 5305 tablet_service.cc:1467] Processing CreateTablet for tablet 2e10b4b1faef440dbb9755d025a2f5fb (DEFAULT_TABLE table=test-table [id=cf13e632697440f98d573eb39d562779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:11.446830 5166 tablet_service.cc:1467] Processing CreateTablet for tablet 2e10b4b1faef440dbb9755d025a2f5fb (DEFAULT_TABLE table=test-table [id=cf13e632697440f98d573eb39d562779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:11.447993 5305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e10b4b1faef440dbb9755d025a2f5fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:11.448143 5166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e10b4b1faef440dbb9755d025a2f5fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:11.450654 5448 tablet_service.cc:1467] Processing CreateTablet for tablet 2e10b4b1faef440dbb9755d025a2f5fb (DEFAULT_TABLE table=test-table [id=cf13e632697440f98d573eb39d562779]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:11.452029 5448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e10b4b1faef440dbb9755d025a2f5fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:11.459412 5535 tablet_bootstrap.cc:492] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c: Bootstrap starting.
I20260809 00:59:11.462309 5535 tablet_bootstrap.cc:654] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:11.463654 5535 log.cc:824] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:11.463851 5536 tablet_bootstrap.cc:492] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb: Bootstrap starting.
I20260809 00:59:11.467278 5535 tablet_bootstrap.cc:492] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c: No bootstrap required, opened a new log
I20260809 00:59:11.467626 5535 ts_tablet_manager.cc:1397] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260809 00:59:11.468051 5536 tablet_bootstrap.cc:654] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:11.469367 5536 log.cc:824] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:11.471824 5538 tablet_bootstrap.cc:492] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2: Bootstrap starting.
I20260809 00:59:11.474253 5536 tablet_bootstrap.cc:492] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb: No bootstrap required, opened a new log
I20260809 00:59:11.474545 5536 ts_tablet_manager.cc:1397] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 00:59:11.475577 5538 tablet_bootstrap.cc:654] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:11.476884 5538 log.cc:824] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:11.478991 5535 raft_consensus.cc:348] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 42423 } } peers { permanent_uuid: "05f19ede507d4456846711db330febd2" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44813 } } peers { permanent_uuid: "ef3f8c7b22164707856193e75a6729bb" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 35839 } }
I20260809 00:59:11.479517 5535 raft_consensus.cc:374] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:11.479640 5535 raft_consensus.cc:729] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87eb9a9c7c6b49f29f0a5fc550f4635c, State: Initialized, Role: FOLLOWER
I20260809 00:59:11.480577 5538 tablet_bootstrap.cc:492] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2: No bootstrap required, opened a new log
I20260809 00:59:11.480453 5535 consensus_queue.cc:260] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 42423 } } peers { permanent_uuid: "05f19ede507d4456846711db330febd2" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44813 } } peers { permanent_uuid: "ef3f8c7b22164707856193e75a6729bb" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 35839 } }
I20260809 00:59:11.480856 5538 ts_tablet_manager.cc:1397] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 00:59:11.483064 5536 raft_consensus.cc:348] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 42423 } } peers { permanent_uuid: "05f19ede507d4456846711db330febd2" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44813 } } peers { permanent_uuid: "ef3f8c7b22164707856193e75a6729bb" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 35839 } }
I20260809 00:59:11.483623 5536 raft_consensus.cc:374] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:11.483763 5536 raft_consensus.cc:729] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef3f8c7b22164707856193e75a6729bb, State: Initialized, Role: FOLLOWER
I20260809 00:59:11.484395 5536 consensus_queue.cc:260] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 42423 } } peers { permanent_uuid: "05f19ede507d4456846711db330febd2" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44813 } } peers { permanent_uuid: "ef3f8c7b22164707856193e75a6729bb" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 35839 } }
I20260809 00:59:11.485704 5374 heartbeater.cc:503] Master 127.4.82.190:37933 was elected leader, sending a full tablet report...
I20260809 00:59:11.486122 5536 ts_tablet_manager.cc:1428] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 00:59:11.488811 5535 ts_tablet_manager.cc:1428] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c: Time spent starting tablet: real 0.021s user 0.011s sys 0.008s
I20260809 00:59:11.493708 5538 raft_consensus.cc:348] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 42423 } } peers { permanent_uuid: "05f19ede507d4456846711db330febd2" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44813 } } peers { permanent_uuid: "ef3f8c7b22164707856193e75a6729bb" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 35839 } }
I20260809 00:59:11.494081 5538 raft_consensus.cc:374] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:11.494215 5538 raft_consensus.cc:729] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05f19ede507d4456846711db330febd2, State: Initialized, Role: FOLLOWER
I20260809 00:59:11.494865 5538 consensus_queue.cc:260] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 42423 } } peers { permanent_uuid: "05f19ede507d4456846711db330febd2" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44813 } } peers { permanent_uuid: "ef3f8c7b22164707856193e75a6729bb" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 35839 } }
I20260809 00:59:11.496043 5517 heartbeater.cc:503] Master 127.4.82.190:37933 was elected leader, sending a full tablet report...
I20260809 00:59:11.496685 5538 ts_tablet_manager.cc:1428] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260809 00:59:11.604279 5518 tablet.cc:2369] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:59:11.626927 5375 tablet.cc:2369] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:59:11.641373 5232 tablet.cc:2369] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:59:11.684787 5544 raft_consensus.cc:482] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:59:11.685050 5544 raft_consensus.cc:504] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 42423 } } peers { permanent_uuid: "05f19ede507d4456846711db330febd2" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44813 } } peers { permanent_uuid: "ef3f8c7b22164707856193e75a6729bb" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 35839 } }
I20260809 00:59:11.686741 5544 leader_election.cc:288] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87eb9a9c7c6b49f29f0a5fc550f4635c (127.4.82.129:42423), ef3f8c7b22164707856193e75a6729bb (127.4.82.130:35839)
I20260809 00:59:11.693665 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.687160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:42423 (local address 127.4.82.131:54739)
0809 00:59:11.687455 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:11.687478 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:11.687609 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:11.688004 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:11.688009 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:11.688034 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:11.688284 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:11.688296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.689634 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.689638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:11.691697 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:11.691714 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.693048 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.693051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:11.693189 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.693212 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 00:59:11.693286 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:59:11.693374 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":22}
I20260809 00:59:11.693737 5531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.687219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:54739 (local address 127.4.82.129:42423)
0809 00:59:11.687490 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:11.687499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:11.687531 (+ 32us) server_negotiation.cc:407] Connection header received
0809 00:59:11.687777 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:11.687785 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:11.687872 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:11.687985 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:11.688452 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.689489 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.691855 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.692910 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.693074 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.693152 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:11.693330 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 00:59:11.693454 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260809 00:59:11.694324 5534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.687271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:36851 (local address 127.4.82.130:35839)
0809 00:59:11.687507 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:11.687514 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:11.688206 (+ 692us) server_negotiation.cc:407] Connection header received
0809 00:59:11.688322 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:11.688360 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:11.688443 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:11.688534 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:11.689101 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.690030 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.692272 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.693436 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.693594 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.693658 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:11.694048 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 00:59:11.694149 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260809 00:59:11.694561 5545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.687219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:35839 (local address 127.4.82.131:36851)
0809 00:59:11.688095 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:11.688110 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:11.688219 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:11.688603 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:11.688606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:11.688625 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:11.688944 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:11.688951 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.690183 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.690189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:11.692116 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:11.692126 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.693650 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.693657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:11.693844 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.693874 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 00:59:11.694008 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 00:59:11.694145 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":1,"thread_start_us":499,"threads_started":1}
I20260809 00:59:11.694895 5186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e10b4b1faef440dbb9755d025a2f5fb" candidate_uuid: "05f19ede507d4456846711db330febd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c" is_pre_election: true
I20260809 00:59:11.695438 5186 raft_consensus.cc:2454] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05f19ede507d4456846711db330febd2 in term 0.
I20260809 00:59:11.695550 5327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e10b4b1faef440dbb9755d025a2f5fb" candidate_uuid: "05f19ede507d4456846711db330febd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef3f8c7b22164707856193e75a6729bb" is_pre_election: true
I20260809 00:59:11.696101 5327 raft_consensus.cc:2454] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05f19ede507d4456846711db330febd2 in term 0.
I20260809 00:59:11.696137 5397 leader_election.cc:302] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05f19ede507d4456846711db330febd2, 87eb9a9c7c6b49f29f0a5fc550f4635c; no voters:
I20260809 00:59:11.696513 5544 raft_consensus.cc:2781] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:59:11.696661 5544 raft_consensus.cc:482] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:59:11.696782 5544 raft_consensus.cc:3037] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:11.699023 5544 raft_consensus.cc:504] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 42423 } } peers { permanent_uuid: "05f19ede507d4456846711db330febd2" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44813 } } peers { permanent_uuid: "ef3f8c7b22164707856193e75a6729bb" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 35839 } }
I20260809 00:59:11.699669 5544 leader_election.cc:288] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [CANDIDATE]: Term 1 election: Requested vote from peers 87eb9a9c7c6b49f29f0a5fc550f4635c (127.4.82.129:42423), ef3f8c7b22164707856193e75a6729bb (127.4.82.130:35839)
I20260809 00:59:11.700029 5186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e10b4b1faef440dbb9755d025a2f5fb" candidate_uuid: "05f19ede507d4456846711db330febd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c"
I20260809 00:59:11.700150 5327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e10b4b1faef440dbb9755d025a2f5fb" candidate_uuid: "05f19ede507d4456846711db330febd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef3f8c7b22164707856193e75a6729bb"
I20260809 00:59:11.700244 5186 raft_consensus.cc:3037] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:11.700395 5327 raft_consensus.cc:3037] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:11.703562 5327 raft_consensus.cc:2454] T 2e10b4b1faef440dbb9755d025a2f5fb P ef3f8c7b22164707856193e75a6729bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05f19ede507d4456846711db330febd2 in term 1.
I20260809 00:59:11.703562 5186 raft_consensus.cc:2454] T 2e10b4b1faef440dbb9755d025a2f5fb P 87eb9a9c7c6b49f29f0a5fc550f4635c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05f19ede507d4456846711db330febd2 in term 1.
I20260809 00:59:11.703961 5399 leader_election.cc:302] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05f19ede507d4456846711db330febd2, ef3f8c7b22164707856193e75a6729bb; no voters:
I20260809 00:59:11.704262 5544 raft_consensus.cc:2781] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:11.704720 5544 raft_consensus.cc:686] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [term 1 LEADER]: Becoming Leader. State: Replica: 05f19ede507d4456846711db330febd2, State: Running, Role: LEADER
I20260809 00:59:11.705173 5544 consensus_queue.cc:237] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 42423 } } peers { permanent_uuid: "05f19ede507d4456846711db330febd2" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44813 } } peers { permanent_uuid: "ef3f8c7b22164707856193e75a6729bb" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 35839 } }
I20260809 00:59:11.709743 5035 catalog_manager.cc:5495] T 2e10b4b1faef440dbb9755d025a2f5fb P 05f19ede507d4456846711db330febd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05f19ede507d4456846711db330febd2 (127.4.82.131). New cstate: current_term: 1 leader_uuid: "05f19ede507d4456846711db330febd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87eb9a9c7c6b49f29f0a5fc550f4635c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 42423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05f19ede507d4456846711db330febd2" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef3f8c7b22164707856193e75a6729bb" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 35839 } health_report { overall_health: UNKNOWN } } }
Aug 09 00:59:11 dist-test-slave-07c2 krb5kdc[4989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 00:59:11.742755 5531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.728446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43210 (local address 127.4.82.129:42423)
0809 00:59:11.728682 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:11.728692 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:11.729145 (+ 453us) server_negotiation.cc:407] Connection header received
0809 00:59:11.729280 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:11.729284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:11.729366 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:11.729451 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:11.730914 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.731763 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.733206 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.733636 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.733733 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.737744 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:11.737756 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:11.737770 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:11.737802 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:11.740421 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.741013 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.741020 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.741022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.741146 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.741440 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.741444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.741446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.741765 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:11.741980 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:11.742233 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 00:59:11.742380 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 00:59:11 dist-test-slave-07c2 krb5kdc[4989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237151, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:59:11 dist-test-slave-07c2 krb5kdc[4989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237151, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 00:59:11.790684 5531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.776254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43214 (local address 127.4.82.129:42423)
0809 00:59:11.776453 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:11.776462 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:11.776811 (+ 349us) server_negotiation.cc:407] Connection header received
0809 00:59:11.777028 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:11.777033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:11.777100 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:11.777214 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:11.778707 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.779529 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.780816 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.781234 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.781373 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.785300 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:11.785305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:11.785309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:11.785341 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:11.787990 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.788726 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.788730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.788732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.788855 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.789304 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.789307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.789309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.789562 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:11.789762 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:11.790149 (+ 387us) server_negotiation.cc:299] Negotiation successful
0809 00:59:11.790319 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 00:59:11 dist-test-slave-07c2 krb5kdc[4989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:59:11 dist-test-slave-07c2 krb5kdc[4989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 00:59:11.838490 5531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.824474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43216 (local address 127.4.82.129:42423)
0809 00:59:11.824676 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:11.824683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:11.825038 (+ 355us) server_negotiation.cc:407] Connection header received
0809 00:59:11.825252 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:11.825257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:11.825325 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:11.825433 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:11.826897 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.827729 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.828964 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.829405 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.829540 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.833408 (+ 3868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:11.833413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:11.833416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:11.833448 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:11.835926 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.836560 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.836566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.836570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.836698 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.837053 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.837059 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.837063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.837365 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:11.837636 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:11.837916 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 00:59:11.838172 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 00:59:11.840253 5144 tablet_service.cc:2908] Rejecting scan request for tablet 2e10b4b1faef440dbb9755d025a2f5fb: Uninitialized: clean time has not yet been initialized
I20260809 00:59:11.843145 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 5098
I20260809 00:59:11.854645 5531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.844108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43226 (local address 127.4.82.129:42423)
0809 00:59:11.844253 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:11.844257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:11.844662 (+ 405us) server_negotiation.cc:407] Connection header received
0809 00:59:11.844783 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:11.844786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:11.844850 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:11.844922 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:11.845966 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.846589 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.847764 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.848174 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.848274 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.850236 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:11.850240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:11.850243 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:11.850270 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:11.852641 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.853239 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.853242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.853244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.853354 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.853683 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.853686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.853688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.853904 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:11.854143 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:11.854291 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 00:59:11.854440 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 00:59:11.855276 5226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:11.954293 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 5098
I20260809 00:59:11.973793 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 5237
Aug 09 00:59:11 dist-test-slave-07c2 krb5kdc[4989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 00:59:11.987993 5534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:11.974603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48382 (local address 127.4.82.130:35839)
0809 00:59:11.974806 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:11.974811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:11.974833 (+ 22us) server_negotiation.cc:407] Connection header received
0809 00:59:11.974908 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:11.974918 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:11.974980 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:11.975100 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:11.976319 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.977052 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.978316 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:11.978857 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:11.979000 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:11.982829 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:11.982843 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:11.982857 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:11.982886 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:11.985531 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.986156 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.986163 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.986165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.986285 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:11.986600 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:11.986604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:11.986607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:11.987078 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:11.987348 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:11.987586 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 00:59:11.987741 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 00:59:11.988689 5369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:12.075541 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 5237
I20260809 00:59:12.094980 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 5379
Aug 09 00:59:12 dist-test-slave-07c2 krb5kdc[4989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 00:59:12.109361 5564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:12.095798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56146 (local address 127.4.82.131:44813)
0809 00:59:12.096231 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:12.096235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:12.096251 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:59:12.096303 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:12.096307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:12.096363 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:12.096460 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:12.097619 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:12.098293 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:12.099559 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:12.099969 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:12.100059 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:12.104213 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:12.104229 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:12.104246 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:12.104314 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:12.107003 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:12.107676 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:12.107684 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:12.107687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:12.107832 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:12.108157 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:12.108160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:12.108162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:12.108484 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:12.108727 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:12.108905 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 00:59:12.109064 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":222,"threads_started":1}
I20260809 00:59:12.109992 5511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:12.199790 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 5379
I20260809 00:59:12.218005 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 5002
I20260809 00:59:12.219141 5066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:12.328142 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 5002
2026-08-09T00:59:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4217 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:59:12 dist-test-slave-07c2 krb5kdc[5571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:59:12 dist-test-slave-07c2 krb5kdc[5571](info): set up 2 sockets
Aug 09 00:59:12 dist-test-slave-07c2 krb5kdc[5571](info): commencing operation
krb5kdc: starting...
W20260809 00:59:13.434368 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.002s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 00:59:13 dist-test-slave-07c2 krb5kdc[5571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:59:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 00:59:13.621188 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:46197
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:46197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:13.934521 5584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:13.934882 5584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:13.935053 5584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:13.942126 5584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:13.942219 5584 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 00:59:13.942277 5584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:13.942358 5584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:13.942430 5584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:13.942509 5584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:13.952309 5584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:46197
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:46197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:13.953552 5584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:13.954886 5584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:13.966537 5590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:13.966598 5592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:13.966816 5594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:13.967597 5584 server_base.cc:1034] running on GCE node
I20260809 00:59:13.968322 5584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:13.969609 5584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:13.970853 5584 hybrid_clock.cc:648] HybridClock initialized: now 1786237153970788 us; error 42 us; skew 500 ppm
I20260809 00:59:13.971179 5584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:13 dist-test-slave-07c2 krb5kdc[5571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237153, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:13.974313 5584 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 00:59:13.975739 5584 webserver.cc:460] Webserver started at http://127.4.82.190:44001/ using document root <none> and password file <none>
I20260809 00:59:13.976430 5584 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:13.976557 5584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:13.976877 5584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:13.979761 5584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "4334c06dc380492487ad62b7cdd5a2cf"
format_stamp: "Formatted at 2026-08-09 00:59:13 on dist-test-slave-07c2"
server_key: "e9a0adf48ff55a0d2cea7611d048412f"
server_key_iv: "2793b7f448e5a2626d1e7be703af2eef"
server_key_version: "encryptionkey@0"
I20260809 00:59:13.980542 5584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "4334c06dc380492487ad62b7cdd5a2cf"
format_stamp: "Formatted at 2026-08-09 00:59:13 on dist-test-slave-07c2"
server_key: "e9a0adf48ff55a0d2cea7611d048412f"
server_key_iv: "2793b7f448e5a2626d1e7be703af2eef"
server_key_version: "encryptionkey@0"
I20260809 00:59:13.985340 5584 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 00:59:13.988525 5602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:13.989670 5584 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:59:13.989845 5584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "4334c06dc380492487ad62b7cdd5a2cf"
format_stamp: "Formatted at 2026-08-09 00:59:13 on dist-test-slave-07c2"
server_key: "e9a0adf48ff55a0d2cea7611d048412f"
server_key_iv: "2793b7f448e5a2626d1e7be703af2eef"
server_key_version: "encryptionkey@0"
I20260809 00:59:13.990051 5584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:14.005645 5584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:14.009985 5584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:14.010249 5584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:14.031443 5584 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:46197
I20260809 00:59:14.031427 5670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:46197 every 8 connection(s)
I20260809 00:59:14.032763 5584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:14.036448 5671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:14.036736 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 5584
I20260809 00:59:14.036966 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:14.037815 4426 external_mini_cluster.cc:1428] Setting key c38a87dea5df702706c05c3bfa626b05
I20260809 00:59:14.050132 5671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf: Bootstrap starting.
Aug 09 00:59:14 dist-test-slave-07c2 krb5kdc[5571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:14.053646 5671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:14.054823 5671 log.cc:824] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:14.057533 5671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf: No bootstrap required, opened a new log
I20260809 00:59:14.060372 5674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:14.039855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57610 (local address 127.4.82.190:46197)
0809 00:59:14.040641 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:14.040652 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:14.040691 (+ 39us) server_negotiation.cc:407] Connection header received
0809 00:59:14.042628 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:14.042648 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:14.042998 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:14.043342 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:14.044966 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:14.046153 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:14.047962 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:14.048563 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:14.048699 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:14.053411 (+ 4712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:14.053418 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:14.053431 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:14.053474 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:14.056518 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:14.057148 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:14.057154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:14.057159 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:14.057317 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:14.057720 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:14.057724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:14.057725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:14.058246 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:14.058560 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:14.058986 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 00:59:14.059432 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":367,"threads_started":1}
I20260809 00:59:14.066277 5671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4334c06dc380492487ad62b7cdd5a2cf" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46197 } }
I20260809 00:59:14.066614 5671 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:14.066740 5671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4334c06dc380492487ad62b7cdd5a2cf, State: Initialized, Role: FOLLOWER
I20260809 00:59:14.067350 5671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4334c06dc380492487ad62b7cdd5a2cf" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46197 } }
I20260809 00:59:14.067595 5671 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:14.067759 5671 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:14.067929 5671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:14.070217 5671 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4334c06dc380492487ad62b7cdd5a2cf" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46197 } }
I20260809 00:59:14.070703 5671 leader_election.cc:302] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4334c06dc380492487ad62b7cdd5a2cf; no voters:
I20260809 00:59:14.071269 5671 leader_election.cc:288] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:14.071414 5676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:14.072240 5676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [term 1 LEADER]: Becoming Leader. State: Replica: 4334c06dc380492487ad62b7cdd5a2cf, State: Running, Role: LEADER
I20260809 00:59:14.072770 5676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4334c06dc380492487ad62b7cdd5a2cf" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46197 } }
I20260809 00:59:14.073491 5671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:14.075675 5677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4334c06dc380492487ad62b7cdd5a2cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4334c06dc380492487ad62b7cdd5a2cf" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46197 } } }
I20260809 00:59:14.076136 5677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:14.076938 5678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4334c06dc380492487ad62b7cdd5a2cf. Latest consensus state: current_term: 1 leader_uuid: "4334c06dc380492487ad62b7cdd5a2cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4334c06dc380492487ad62b7cdd5a2cf" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46197 } } }
I20260809 00:59:14.077286 5678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:14.079991 5682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:14.084504 5682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:14.092556 5682 catalog_manager.cc:1347] Generated new cluster ID: b94ccacc2c594c75bf0f7f0ee7ddbd5a
I20260809 00:59:14.092665 5682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:14.135522 5682 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:14.137485 5682 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:14.151430 5682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf: Generated new TSK 0
I20260809 00:59:14.152376 5682 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 00:59:14.200591 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46197
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:14.518072 5701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:14.518332 5701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:14.518496 5701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:14.525707 5701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:14.525833 5701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:14.525988 5701 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:59:14.526093 5701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 00:59:14.535670 5701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.82.190:46197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:14.536943 5701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:14.538318 5701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:14.551409 5707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:14.551349 5708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:14.551355 5710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:14.552287 5701 server_base.cc:1034] running on GCE node
I20260809 00:59:14.552930 5701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:14.553788 5701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:14.555011 5701 hybrid_clock.cc:648] HybridClock initialized: now 1786237154554922 us; error 51 us; skew 500 ppm
I20260809 00:59:14.555372 5701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:14 dist-test-slave-07c2 krb5kdc[5571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237154, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:14.558645 5701 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 00:59:14.560062 5701 webserver.cc:460] Webserver started at http://127.4.82.129:44553/ using document root <none> and password file <none>
I20260809 00:59:14.560721 5701 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:14.560853 5701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:14.561218 5701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:14.564082 5701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "64d565dacb8346c7bd7f67a6f17a2e5c"
format_stamp: "Formatted at 2026-08-09 00:59:14 on dist-test-slave-07c2"
server_key: "339d864d1532c506255d949e37bb2d1a"
server_key_iv: "dc0398098881ce1083fa474a65798732"
server_key_version: "encryptionkey@0"
I20260809 00:59:14.564823 5701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "64d565dacb8346c7bd7f67a6f17a2e5c"
format_stamp: "Formatted at 2026-08-09 00:59:14 on dist-test-slave-07c2"
server_key: "339d864d1532c506255d949e37bb2d1a"
server_key_iv: "dc0398098881ce1083fa474a65798732"
server_key_version: "encryptionkey@0"
I20260809 00:59:14.569787 5701 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 00:59:14.573086 5718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:14.574569 5701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 00:59:14.574738 5701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "64d565dacb8346c7bd7f67a6f17a2e5c"
format_stamp: "Formatted at 2026-08-09 00:59:14 on dist-test-slave-07c2"
server_key: "339d864d1532c506255d949e37bb2d1a"
server_key_iv: "dc0398098881ce1083fa474a65798732"
server_key_version: "encryptionkey@0"
I20260809 00:59:14.574935 5701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:14.603086 5701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:14.607584 5701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:14.607846 5701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:14.610564 5701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:14.610673 5701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:14.610844 5701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:14.610927 5701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:59:14 dist-test-slave-07c2 krb5kdc[5571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237154, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:14.638105 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:14.611282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:56849 (local address 127.4.82.190:46197)
0809 00:59:14.611879 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:14.611885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:14.613029 (+ 1144us) server_negotiation.cc:407] Connection header received
0809 00:59:14.615019 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:14.615025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:14.615113 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:14.615249 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:14.617388 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:14.618340 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:14.621542 (+ 3202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:14.622084 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:14.622228 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:14.627308 (+ 5080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:14.627319 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:14.627327 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:14.627382 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:14.631102 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:14.632386 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:14.632393 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:14.632400 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:14.632570 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:14.633161 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:14.633167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:14.633174 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:14.633600 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:14.633788 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:14.637357 (+ 3569us) server_negotiation.cc:299] Negotiation successful
0809 00:59:14.637629 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":333,"threads_started":1}
I20260809 00:59:14.638770 5730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:14.611944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:46197 (local address 127.4.82.129:56849)
0809 00:59:14.612843 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:14.612895 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:14.614753 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:14.615542 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:14.615555 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:14.616353 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:14.617214 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:14.617234 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:14.618536 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:14.618545 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:14.621373 (+ 2828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:14.621390 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:14.622358 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:14.622370 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:14.622787 (+ 417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:14.623663 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:14.623726 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:14.627028 (+ 3302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:14.631247 (+ 4219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:14.631262 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:14.631281 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:14.632214 (+ 933us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:14.632710 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:14.632714 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:14.632716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:14.632999 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:14.633882 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:14.633891 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:14.634482 (+ 591us) client_negotiation.cc:769] Sending connection context
0809 00:59:14.634852 (+ 370us) client_negotiation.cc:240] Negotiation successful
0809 00:59:14.635579 (+ 727us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":390,"threads_started":1}
I20260809 00:59:14.656374 5701 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:37771
I20260809 00:59:14.656412 5846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:37771 every 8 connection(s)
I20260809 00:59:14.657992 5701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 00:59:14.659518 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:14.645797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:41019 (local address 127.4.82.190:46197)
0809 00:59:14.645984 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:14.645992 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:14.646565 (+ 573us) server_negotiation.cc:407] Connection header received
0809 00:59:14.646828 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:14.646833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:14.646903 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:14.646955 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:14.648471 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:14.649199 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:14.650511 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:14.650953 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:14.651106 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:14.653790 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:14.653803 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:14.653811 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:14.653870 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:14.656671 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:14.657398 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:14.657402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:14.657404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:14.657526 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:14.657911 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:14.657914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:14.657916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:14.658209 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:14.658318 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:14.658896 (+ 578us) server_negotiation.cc:299] Negotiation successful
0809 00:59:14.659130 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 00:59:14.659896 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:14.645820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:46197 (local address 127.4.82.129:41019)
0809 00:59:14.646411 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:14.646455 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:14.646617 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:14.647172 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:14.647178 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:14.647862 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:14.648327 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:14.648340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:14.649334 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:14.649341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:14.650353 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:14.650365 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:14.651178 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:14.651188 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:14.651486 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:14.652236 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:14.652267 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:14.653563 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:14.656787 (+ 3224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:14.656792 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:14.656796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:14.657292 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:14.657633 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:14.657638 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:14.657641 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:14.657794 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:14.658322 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:14.658327 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:14.658568 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 00:59:14.658775 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 00:59:14.659232 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":328,"threads_started":1}
I20260809 00:59:14.660128 5847 heartbeater.cc:347] Connected to a master server at 127.4.82.190:46197
I20260809 00:59:14.660369 5847 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:14.661248 5847 heartbeater.cc:511] Master 127.4.82.190:46197 requested a full tablet report, sending...
I20260809 00:59:14.663489 5621 ts_manager.cc:194] Registered new tserver with Master: 64d565dacb8346c7bd7f67a6f17a2e5c (127.4.82.129:37771)
I20260809 00:59:14.664880 5621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:56849
I20260809 00:59:14.667949 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 5701
I20260809 00:59:14.668164 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 00:59:14.668816 4426 external_mini_cluster.cc:1428] Setting key 19b7ac673f18ef2c0f77beb41d910730
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 00:59:14.708689 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46197
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:15.025131 5851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:15.025394 5851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:15.025558 5851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:15.033754 5851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:15.033876 5851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:15.034006 5851 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:59:15.034117 5851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 00:59:15.048291 5851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.82.190:46197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:15.049465 5851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:15.050733 5851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:15.063764 5856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:15.063851 5857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:15.063858 5860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:15.065611 5851 server_base.cc:1034] running on GCE node
I20260809 00:59:15.066201 5851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:15.067060 5851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:15.068310 5851 hybrid_clock.cc:648] HybridClock initialized: now 1786237155068234 us; error 69 us; skew 500 ppm
I20260809 00:59:15.068642 5851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:15 dist-test-slave-07c2 krb5kdc[5571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237155, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:15.077415 5851 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 00:59:15.079042 5851 webserver.cc:460] Webserver started at http://127.4.82.130:38699/ using document root <none> and password file <none>
I20260809 00:59:15.079864 5851 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:15.079954 5851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:15.080282 5851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:15.083195 5851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "2e90e2a989d64b6d8e426c05657eaa12"
format_stamp: "Formatted at 2026-08-09 00:59:15 on dist-test-slave-07c2"
server_key: "918b7404867967b4e9947341583f99d8"
server_key_iv: "8e4dd5b5af03dfda1bd91d8152b51668"
server_key_version: "encryptionkey@0"
I20260809 00:59:15.083997 5851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "2e90e2a989d64b6d8e426c05657eaa12"
format_stamp: "Formatted at 2026-08-09 00:59:15 on dist-test-slave-07c2"
server_key: "918b7404867967b4e9947341583f99d8"
server_key_iv: "8e4dd5b5af03dfda1bd91d8152b51668"
server_key_version: "encryptionkey@0"
I20260809 00:59:15.088977 5851 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 00:59:15.092476 5867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:15.093823 5851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 00:59:15.093972 5851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "2e90e2a989d64b6d8e426c05657eaa12"
format_stamp: "Formatted at 2026-08-09 00:59:15 on dist-test-slave-07c2"
server_key: "918b7404867967b4e9947341583f99d8"
server_key_iv: "8e4dd5b5af03dfda1bd91d8152b51668"
server_key_version: "encryptionkey@0"
I20260809 00:59:15.094148 5851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:15.109730 5851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:15.113372 5851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:15.113575 5851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:15.115993 5851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:15.116125 5851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:15.116326 5851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:15.116391 5851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:59:15 dist-test-slave-07c2 krb5kdc[5571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237155, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:15.137522 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.116389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:53347 (local address 127.4.82.190:46197)
0809 00:59:15.116596 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:15.116603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:15.117877 (+ 1274us) server_negotiation.cc:407] Connection header received
0809 00:59:15.119906 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:15.119913 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:15.120030 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:15.120140 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:15.122179 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.123550 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.126043 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.126479 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.126607 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.130859 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:15.130869 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:15.130877 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:15.130933 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:15.134555 (+ 3622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:15.135272 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:15.135278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:15.135282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:15.135444 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:15.135890 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:15.135896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:15.135899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:15.136166 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:15.136329 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:15.136977 (+ 648us) server_negotiation.cc:299] Negotiation successful
0809 00:59:15.137187 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 00:59:15.138551 5877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.116857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:46197 (local address 127.4.82.130:53347)
0809 00:59:15.117694 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:15.117745 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:15.119621 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:15.120371 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:15.120379 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:15.121101 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:15.121953 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.121991 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.123758 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.123764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.125866 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.125888 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.126724 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.126731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.126975 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.127862 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:15.127897 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:15.130620 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:15.134690 (+ 4070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:15.134698 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:15.134722 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:15.135146 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:15.135557 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:15.135562 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:15.135566 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:15.135760 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:15.136306 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:15.136313 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:15.136625 (+ 312us) client_negotiation.cc:769] Sending connection context
0809 00:59:15.136875 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 00:59:15.137339 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":364,"threads_started":1}
I20260809 00:59:15.158018 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.144340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:33207 (local address 127.4.82.190:46197)
0809 00:59:15.144481 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:15.144485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:15.145174 (+ 689us) server_negotiation.cc:407] Connection header received
0809 00:59:15.145413 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:15.145424 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:15.145485 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:15.145581 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:15.146554 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.147949 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.149795 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.150207 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.150345 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.152258 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:15.152271 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:15.152275 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:15.152302 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:15.154786 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:15.155423 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:15.155431 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:15.155437 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:15.155604 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:15.156097 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:15.156104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:15.156108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:15.156548 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:15.156699 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:15.157309 (+ 610us) server_negotiation.cc:299] Negotiation successful
0809 00:59:15.157585 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:59:15.158208 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.144424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:46197 (local address 127.4.82.130:33207)
0809 00:59:15.145050 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:15.145068 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:15.145266 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:15.145606 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:15.145611 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:15.146046 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:15.146386 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.146400 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.148111 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.148118 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.149646 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.149661 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.150409 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.150416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.150577 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.151231 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:15.151247 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:15.152072 (+ 825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:15.154908 (+ 2836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:15.154914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:15.154918 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:15.155281 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:15.155760 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:15.155764 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:15.155767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:15.155934 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:15.156779 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:15.156785 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:15.156967 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 00:59:15.157157 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 00:59:15.157559 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":362,"threads_started":1}
I20260809 00:59:15.175671 5851 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:42613
I20260809 00:59:15.175706 5993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:42613 every 8 connection(s)
I20260809 00:59:15.176815 5851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 00:59:15.178808 5994 heartbeater.cc:347] Connected to a master server at 127.4.82.190:46197
I20260809 00:59:15.179029 5994 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:15.179827 5994 heartbeater.cc:511] Master 127.4.82.190:46197 requested a full tablet report, sending...
I20260809 00:59:15.181139 5621 ts_manager.cc:194] Registered new tserver with Master: 2e90e2a989d64b6d8e426c05657eaa12 (127.4.82.130:42613)
I20260809 00:59:15.181926 5621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:53347
I20260809 00:59:15.186486 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 5851
I20260809 00:59:15.186684 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 00:59:15.187098 4426 external_mini_cluster.cc:1428] Setting key bba15e2eac534d9ec3be596b7215b3f2
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 00:59:15.227761 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46197
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:15.545838 6000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:15.546144 6000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:15.546321 6000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:15.553818 6000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:15.553941 6000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:15.554025 6000 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:59:15.554198 6000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 00:59:15.564391 6000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.82.190:46197
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:15.565646 6000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:15.566968 6000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:15.583074 6010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:15.583137 6008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:15.583505 6006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:15.583974 6000 server_base.cc:1034] running on GCE node
I20260809 00:59:15.584563 6000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:15.585417 6000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:15.586652 6000 hybrid_clock.cc:648] HybridClock initialized: now 1786237155586593 us; error 37 us; skew 500 ppm
I20260809 00:59:15.586939 6000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:15 dist-test-slave-07c2 krb5kdc[5571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237155, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:15.590104 6000 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 00:59:15.591543 6000 webserver.cc:460] Webserver started at http://127.4.82.131:40955/ using document root <none> and password file <none>
I20260809 00:59:15.592263 6000 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:15.592350 6000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:15.592654 6000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:15.595479 6000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "bf1938eabb154f2eb454acde458b72e1"
format_stamp: "Formatted at 2026-08-09 00:59:15 on dist-test-slave-07c2"
server_key: "0f4e143b9292074f1f11bc15f9c07b44"
server_key_iv: "46d8793a1fc140b3307105e20c73a8af"
server_key_version: "encryptionkey@0"
I20260809 00:59:15.596205 6000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "bf1938eabb154f2eb454acde458b72e1"
format_stamp: "Formatted at 2026-08-09 00:59:15 on dist-test-slave-07c2"
server_key: "0f4e143b9292074f1f11bc15f9c07b44"
server_key_iv: "46d8793a1fc140b3307105e20c73a8af"
server_key_version: "encryptionkey@0"
I20260809 00:59:15.600991 6000 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 00:59:15.604220 6017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:15.605387 6000 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 00:59:15.605510 6000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "bf1938eabb154f2eb454acde458b72e1"
format_stamp: "Formatted at 2026-08-09 00:59:15 on dist-test-slave-07c2"
server_key: "0f4e143b9292074f1f11bc15f9c07b44"
server_key_iv: "46d8793a1fc140b3307105e20c73a8af"
server_key_version: "encryptionkey@0"
I20260809 00:59:15.605684 6000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:15.629426 6000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:15.633577 6000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:15.633859 6000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:15.636363 6000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:15.636473 6000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:15.636646 6000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:15.636734 6000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:59:15 dist-test-slave-07c2 krb5kdc[5571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237155, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:15.663301 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.637018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:49811 (local address 127.4.82.190:46197)
0809 00:59:15.637222 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:15.637228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:15.638624 (+ 1396us) server_negotiation.cc:407] Connection header received
0809 00:59:15.641462 (+ 2838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:15.641472 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:15.641600 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:15.641726 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:15.643907 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.644928 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.648276 (+ 3348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.648675 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.648775 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.654459 (+ 5684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:15.654464 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:15.654467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:15.654498 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:15.658672 (+ 4174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:15.659964 (+ 1292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:15.659972 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:15.659977 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:15.660156 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:15.660846 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:15.660854 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:15.660860 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:15.661342 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:15.661530 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:15.662443 (+ 913us) server_negotiation.cc:299] Negotiation successful
0809 00:59:15.662738 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 00:59:15.663966 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.637557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:46197 (local address 127.4.82.131:49811)
0809 00:59:15.638429 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:15.638477 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:15.640993 (+ 2516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:15.642063 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:15.642073 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:15.642862 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:15.643727 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.643746 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.645125 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.645131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.648104 (+ 2973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.648123 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.648942 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.648955 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.649384 (+ 429us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.650206 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:15.650247 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:15.654211 (+ 3964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:15.658881 (+ 4670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:15.658894 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:15.658917 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:15.659749 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:15.660317 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:15.660323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:15.660330 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:15.660648 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:15.661499 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:15.661507 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:15.661918 (+ 411us) client_negotiation.cc:769] Sending connection context
0809 00:59:15.662225 (+ 307us) client_negotiation.cc:240] Negotiation successful
0809 00:59:15.662781 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":371,"threads_started":1}
I20260809 00:59:15.667486 5847 heartbeater.cc:503] Master 127.4.82.190:46197 was elected leader, sending a full tablet report...
I20260809 00:59:15.684268 6000 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:43661
I20260809 00:59:15.684324 6151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:43661 every 8 connection(s)
I20260809 00:59:15.685738 6000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 00:59:15.688016 6152 heartbeater.cc:347] Connected to a master server at 127.4.82.190:46197
I20260809 00:59:15.688256 6152 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:15.688879 6152 heartbeater.cc:511] Master 127.4.82.190:46197 requested a full tablet report, sending...
I20260809 00:59:15.690303 5621 ts_manager.cc:194] Registered new tserver with Master: bf1938eabb154f2eb454acde458b72e1 (127.4.82.131:43661)
I20260809 00:59:15.691461 5621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:49811
I20260809 00:59:15.693760 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.677994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:36667 (local address 127.4.82.190:46197)
0809 00:59:15.678165 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:15.678169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:15.678961 (+ 792us) server_negotiation.cc:407] Connection header received
0809 00:59:15.679199 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:15.679205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:15.679343 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:15.679453 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:15.680890 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.682202 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.684493 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.685110 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.685263 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.687527 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:15.687532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:15.687535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:15.687564 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:15.690981 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:15.691689 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:15.691695 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:15.691701 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:15.691880 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:15.692273 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:15.692278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:15.692281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:15.692529 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:15.692721 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:15.693172 (+ 451us) server_negotiation.cc:299] Negotiation successful
0809 00:59:15.693400 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 00:59:15.693848 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.678094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:46197 (local address 127.4.82.131:36667)
0809 00:59:15.678854 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:15.678869 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:15.679059 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:15.679541 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:15.679547 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:15.680180 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:15.680704 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.680731 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.682383 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.682388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.684317 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.684334 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.685292 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.685296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.685423 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.686041 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:15.686063 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:15.687346 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:15.691108 (+ 3762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:15.691119 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:15.691122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:15.691559 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:15.692009 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:15.692014 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:15.692017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:15.692154 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:15.692699 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:15.692704 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:15.692877 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 00:59:15.693068 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 00:59:15.693400 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":407,"threads_started":1}
I20260809 00:59:15.694460 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 6000
I20260809 00:59:15.694666 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 00:59:15.695176 4426 external_mini_cluster.cc:1428] Setting key 25643e11b8b82d65353b963fd3ea516e
I20260809 00:59:15.698184 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 00:59:15 dist-test-slave-07c2 krb5kdc[5571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237155, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:59:15 dist-test-slave-07c2 krb5kdc[5571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237155, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:15.744560 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.731339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57616 (local address 127.4.82.190:46197)
0809 00:59:15.731566 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:15.731573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:15.731784 (+ 211us) server_negotiation.cc:407] Connection header received
0809 00:59:15.731930 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:15.731934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:15.732010 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:15.732093 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:15.733403 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.734131 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.735322 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.735763 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.735855 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.739526 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:15.739530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:15.739533 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:15.739562 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:15.742140 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:15.742729 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:15.742733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:15.742735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:15.742843 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:15.743200 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:15.743205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:15.743207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:15.743562 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:15.743767 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:15.743993 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 00:59:15.744155 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 00:59:15.748440 5621 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:59:15.755201 5621 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:59:15.778793 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.766893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35812 (local address 127.4.82.130:42613)
0809 00:59:15.767564 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:15.767573 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:15.768498 (+ 925us) server_negotiation.cc:407] Connection header received
0809 00:59:15.768864 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:15.768874 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:15.769113 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:15.769323 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:15.770255 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.772056 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.775899 (+ 3843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.777233 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.777356 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.777543 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:15.778281 (+ 738us) server_negotiation.cc:299] Negotiation successful
0809 00:59:15.778460 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":286,"threads_started":1}
I20260809 00:59:15.779577 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.766513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:42613 (local address 127.0.0.1:35812)
0809 00:59:15.768236 (+ 1723us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:15.768292 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:15.768491 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:15.769561 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:15.769568 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:15.769596 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:15.770032 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.770042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.772237 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.772242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.775709 (+ 3467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.775733 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.777765 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.777773 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.778017 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.778072 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 00:59:15.778992 (+ 920us) client_negotiation.cc:240] Negotiation successful
0809 00:59:15.779201 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1423,"thread_start_us":2205,"threads_started":1}
I20260809 00:59:15.780447 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.767060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:37771 (local address 127.0.0.1:43866)
0809 00:59:15.769028 (+ 1968us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:15.769046 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:15.769166 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:15.770017 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:15.770022 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:15.770044 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:15.770488 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.770500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.773056 (+ 2556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.773062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.776204 (+ 3142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.776229 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.779617 (+ 3388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.779625 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.779860 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.779891 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 00:59:15.779965 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:59:15.780131 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1843,"thread_start_us":709,"threads_started":1}
I20260809 00:59:15.781196 6168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.767612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43866 (local address 127.4.82.129:37771)
0809 00:59:15.768077 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:15.768085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:15.769304 (+ 1219us) server_negotiation.cc:407] Connection header received
0809 00:59:15.769403 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:15.769410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:15.769623 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:15.769817 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:15.770689 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.772901 (+ 2212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.776390 (+ 3489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.777919 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.778044 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.780863 (+ 2819us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:15.780913 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 00:59:15.781015 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":782,"threads_started":1}
I20260809 00:59:15.784379 5922 tablet_service.cc:1467] Processing CreateTablet for tablet 1911c3a2ee1543238d654b35024e54b0 (DEFAULT_TABLE table=test-table [id=0e1ef063235949d38f771ba9118694b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:15.784629 5774 tablet_service.cc:1467] Processing CreateTablet for tablet 1911c3a2ee1543238d654b35024e54b0 (DEFAULT_TABLE table=test-table [id=0e1ef063235949d38f771ba9118694b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:15.785727 5922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1911c3a2ee1543238d654b35024e54b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:15.785966 5774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1911c3a2ee1543238d654b35024e54b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:15.789225 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.769096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:43661 (local address 127.0.0.1:49608)
0809 00:59:15.779887 (+ 10791us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:15.779904 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:15.779992 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:15.782645 (+ 2653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:15.782650 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:15.782675 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:15.782852 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.782862 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.784781 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.784787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.786735 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.786750 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.788499 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.788505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.788704 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.788747 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 00:59:15.788838 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 00:59:15.788960 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10706,"thread_start_us":12217,"threads_started":1}
I20260809 00:59:15.789356 6170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.769525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49608 (local address 127.4.82.131:43661)
0809 00:59:15.782097 (+ 12572us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:15.782105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:15.782137 (+ 32us) server_negotiation.cc:407] Connection header received
0809 00:59:15.782231 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:15.782238 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:15.782457 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:15.782638 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:15.783040 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.784604 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.786915 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.788332 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.788470 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.788720 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:15.788870 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 00:59:15.789025 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12414,"thread_start_us":12248,"threads_started":1}
I20260809 00:59:15.794147 6073 tablet_service.cc:1467] Processing CreateTablet for tablet 1911c3a2ee1543238d654b35024e54b0 (DEFAULT_TABLE table=test-table [id=0e1ef063235949d38f771ba9118694b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:15.795516 6073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1911c3a2ee1543238d654b35024e54b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:15.802088 6172 tablet_bootstrap.cc:492] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12: Bootstrap starting.
I20260809 00:59:15.805145 6171 tablet_bootstrap.cc:492] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c: Bootstrap starting.
I20260809 00:59:15.805975 6172 tablet_bootstrap.cc:654] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:15.807269 6172 log.cc:824] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:15.808393 6171 tablet_bootstrap.cc:654] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:15.809653 6171 log.cc:824] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:15.810745 6172 tablet_bootstrap.cc:492] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12: No bootstrap required, opened a new log
I20260809 00:59:15.811082 6172 ts_tablet_manager.cc:1397] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260809 00:59:15.812280 6175 tablet_bootstrap.cc:492] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1: Bootstrap starting.
I20260809 00:59:15.814386 6171 tablet_bootstrap.cc:492] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c: No bootstrap required, opened a new log
I20260809 00:59:15.814627 6171 ts_tablet_manager.cc:1397] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 00:59:15.814894 6175 tablet_bootstrap.cc:654] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:15.815887 6175 log.cc:824] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:15.821256 6175 tablet_bootstrap.cc:492] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1: No bootstrap required, opened a new log
I20260809 00:59:15.821506 6175 ts_tablet_manager.cc:1397] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260809 00:59:15.821959 6172 raft_consensus.cc:348] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } }
I20260809 00:59:15.822395 6172 raft_consensus.cc:374] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:15.822495 6172 raft_consensus.cc:729] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e90e2a989d64b6d8e426c05657eaa12, State: Initialized, Role: FOLLOWER
I20260809 00:59:15.823061 6172 consensus_queue.cc:260] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } }
I20260809 00:59:15.825766 6171 raft_consensus.cc:348] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } }
I20260809 00:59:15.826267 6171 raft_consensus.cc:374] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:15.826452 6171 raft_consensus.cc:729] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d565dacb8346c7bd7f67a6f17a2e5c, State: Initialized, Role: FOLLOWER
I20260809 00:59:15.826892 5994 heartbeater.cc:503] Master 127.4.82.190:46197 was elected leader, sending a full tablet report...
I20260809 00:59:15.827088 6171 consensus_queue.cc:260] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } }
I20260809 00:59:15.828190 6172 ts_tablet_manager.cc:1428] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12: Time spent starting tablet: real 0.017s user 0.016s sys 0.001s
I20260809 00:59:15.829901 6171 ts_tablet_manager.cc:1428] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260809 00:59:15.829873 6175 raft_consensus.cc:348] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } }
I20260809 00:59:15.830278 6175 raft_consensus.cc:374] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:15.830417 6175 raft_consensus.cc:729] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf1938eabb154f2eb454acde458b72e1, State: Initialized, Role: FOLLOWER
I20260809 00:59:15.831598 6175 consensus_queue.cc:260] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } }
I20260809 00:59:15.832716 6152 heartbeater.cc:503] Master 127.4.82.190:46197 was elected leader, sending a full tablet report...
I20260809 00:59:15.833402 6175 ts_tablet_manager.cc:1428] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260809 00:59:15.914052 5848 tablet.cc:2369] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:59:15.928812 5995 tablet.cc:2369] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:59:15.931104 6180 raft_consensus.cc:482] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:59:15.931417 6180 raft_consensus.cc:504] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } }
I20260809 00:59:15.932978 6180 leader_election.cc:288] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e90e2a989d64b6d8e426c05657eaa12 (127.4.82.130:42613), 64d565dacb8346c7bd7f67a6f17a2e5c (127.4.82.129:37771)
W20260809 00:59:15.937713 6153 tablet.cc:2369] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:59:15.939325 6168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.933522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:44609 (local address 127.4.82.129:37771)
0809 00:59:15.933759 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:15.933763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:15.933777 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:59:15.933834 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:15.933837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:15.933902 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:15.934014 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:15.934556 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.935626 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.937582 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.938423 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.938592 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.938669 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:15.938928 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 00:59:15.939050 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 00:59:15.939324 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.933238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:37771 (local address 127.4.82.131:44609)
0809 00:59:15.933422 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:15.933439 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:15.933537 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:15.934089 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:15.934097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:15.934151 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:15.934399 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.934409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.935776 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.935782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.937410 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.937422 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.938577 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.938583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.938772 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.938796 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 00:59:15.938901 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 00:59:15.939028 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 00:59:15.940603 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.933249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:35365 (local address 127.4.82.130:42613)
0809 00:59:15.933461 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:15.933468 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:15.934563 (+ 1095us) server_negotiation.cc:407] Connection header received
0809 00:59:15.934747 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:15.934751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:15.934806 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:15.934869 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:15.935494 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.936532 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.938689 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.939634 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.939817 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.939885 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:15.940192 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 00:59:15.940349 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 00:59:15.940503 5796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1911c3a2ee1543238d654b35024e54b0" candidate_uuid: "bf1938eabb154f2eb454acde458b72e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" is_pre_election: true
I20260809 00:59:15.940831 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:15.933718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:42613 (local address 127.4.82.131:35365)
0809 00:59:15.934441 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:15.934454 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:15.934602 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:15.934914 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:15.934918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:15.934945 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:15.935339 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.935352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.936671 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.936674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.938523 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:15.938535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:15.939826 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:15.939831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:15.940032 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:15.940065 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 00:59:15.940180 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 00:59:15.940310 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":2,"thread_start_us":407,"threads_started":1}
I20260809 00:59:15.941098 5796 raft_consensus.cc:2454] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf1938eabb154f2eb454acde458b72e1 in term 0.
I20260809 00:59:15.941787 6018 leader_election.cc:302] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d565dacb8346c7bd7f67a6f17a2e5c, bf1938eabb154f2eb454acde458b72e1; no voters:
I20260809 00:59:15.941901 5948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1911c3a2ee1543238d654b35024e54b0" candidate_uuid: "bf1938eabb154f2eb454acde458b72e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e90e2a989d64b6d8e426c05657eaa12" is_pre_election: true
I20260809 00:59:15.942212 6180 raft_consensus.cc:2781] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:59:15.942373 6180 raft_consensus.cc:482] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:59:15.942497 6180 raft_consensus.cc:3037] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:15.942523 5948 raft_consensus.cc:2454] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf1938eabb154f2eb454acde458b72e1 in term 0.
I20260809 00:59:15.944936 6180 raft_consensus.cc:504] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } }
I20260809 00:59:15.945432 6180 leader_election.cc:288] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [CANDIDATE]: Term 1 election: Requested vote from peers 2e90e2a989d64b6d8e426c05657eaa12 (127.4.82.130:42613), 64d565dacb8346c7bd7f67a6f17a2e5c (127.4.82.129:37771)
I20260809 00:59:15.945858 5948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1911c3a2ee1543238d654b35024e54b0" candidate_uuid: "bf1938eabb154f2eb454acde458b72e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e90e2a989d64b6d8e426c05657eaa12"
I20260809 00:59:15.945878 5796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1911c3a2ee1543238d654b35024e54b0" candidate_uuid: "bf1938eabb154f2eb454acde458b72e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c"
I20260809 00:59:15.946051 5948 raft_consensus.cc:3037] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:15.946063 5796 raft_consensus.cc:3037] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:15.948369 5796 raft_consensus.cc:2454] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf1938eabb154f2eb454acde458b72e1 in term 1.
I20260809 00:59:15.948585 5948 raft_consensus.cc:2454] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf1938eabb154f2eb454acde458b72e1 in term 1.
I20260809 00:59:15.948814 6018 leader_election.cc:302] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d565dacb8346c7bd7f67a6f17a2e5c, bf1938eabb154f2eb454acde458b72e1; no voters:
I20260809 00:59:15.949100 6180 raft_consensus.cc:2781] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:15.949528 6180 raft_consensus.cc:686] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [term 1 LEADER]: Becoming Leader. State: Replica: bf1938eabb154f2eb454acde458b72e1, State: Running, Role: LEADER
I20260809 00:59:15.949981 6180 consensus_queue.cc:237] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } }
I20260809 00:59:15.954663 5619 catalog_manager.cc:5495] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf1938eabb154f2eb454acde458b72e1 (127.4.82.131). New cstate: current_term: 1 leader_uuid: "bf1938eabb154f2eb454acde458b72e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } health_report { overall_health: UNKNOWN } } }
I20260809 00:59:16.020162 6170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:16.015116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49618 (local address 127.4.82.131:43661)
0809 00:59:16.015330 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:16.015335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:16.015480 (+ 145us) server_negotiation.cc:407] Connection header received
0809 00:59:16.015720 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:16.015724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:16.015798 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:16.015914 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:59:16.016226 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.016933 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.018216 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.018634 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.018733 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:16.019162 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:59:16.019669 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:59:16.019777 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:16.019904 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 00:59:16.019966 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 00:59:16.027637 5948 raft_consensus.cc:1261] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12 [term 1 FOLLOWER]: Refusing update from remote peer bf1938eabb154f2eb454acde458b72e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:16.027637 5796 raft_consensus.cc:1261] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 1 FOLLOWER]: Refusing update from remote peer bf1938eabb154f2eb454acde458b72e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:16.028667 6182 consensus_queue.cc:1035] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:16.028996 6180 consensus_queue.cc:1035] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:16.041364 6186 mvcc.cc:205] Tried to move back new op lower bound from 7316427391079469056 to 7316427390774509568. Current Snapshot: MvccSnapshot[applied={T|T < 7316427391079469056}]
I20260809 00:59:16.064674 5619 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:16.064936 5619 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:16.068704 5619 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf: Sending DeleteTablet for 3 replicas of tablet 1911c3a2ee1543238d654b35024e54b0
I20260809 00:59:16.069824 5922 tablet_service.cc:1514] Processing DeleteTablet for tablet 1911c3a2ee1543238d654b35024e54b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:16 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:35812
I20260809 00:59:16.069836 5774 tablet_service.cc:1514] Processing DeleteTablet for tablet 1911c3a2ee1543238d654b35024e54b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:16 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:43866
I20260809 00:59:16.070067 6073 tablet_service.cc:1514] Processing DeleteTablet for tablet 1911c3a2ee1543238d654b35024e54b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:16 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:49608
I20260809 00:59:16.071430 5619 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:59:16.072446 5619 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:59:16.078372 6196 tablet_replica.cc:330] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12: stopping tablet replica
I20260809 00:59:16.078574 6195 tablet_replica.cc:330] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c: stopping tablet replica
I20260809 00:59:16.078886 6195 raft_consensus.cc:2229] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:16.078862 6196 raft_consensus.cc:2229] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:16.079401 6195 raft_consensus.cc:2258] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:16.079443 6196 raft_consensus.cc:2258] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:16.080201 6197 tablet_replica.cc:330] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1: stopping tablet replica
I20260809 00:59:16.080566 6197 raft_consensus.cc:2229] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:59:16.081231 6197 raft_consensus.cc:2258] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:16.081704 6196 ts_tablet_manager.cc:1905] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:16.083185 6073 tablet_service.cc:1467] Processing CreateTablet for tablet d3fa0be13f0b4258962f8b4edb7d3415 (DEFAULT_TABLE table=test-table [id=d5736664ec254d1986f45a89838ff428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:16.082975 5922 tablet_service.cc:1467] Processing CreateTablet for tablet d3fa0be13f0b4258962f8b4edb7d3415 (DEFAULT_TABLE table=test-table [id=d5736664ec254d1986f45a89838ff428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:16.083333 5774 tablet_service.cc:1467] Processing CreateTablet for tablet d3fa0be13f0b4258962f8b4edb7d3415 (DEFAULT_TABLE table=test-table [id=d5736664ec254d1986f45a89838ff428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:16.084124 5922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3fa0be13f0b4258962f8b4edb7d3415. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:16.084235 5774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3fa0be13f0b4258962f8b4edb7d3415. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:16.084522 6195 ts_tablet_manager.cc:1905] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:16.089483 6073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3fa0be13f0b4258962f8b4edb7d3415. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:16.090972 6172 tablet_bootstrap.cc:492] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12: Bootstrap starting.
I20260809 00:59:16.092612 6172 tablet_bootstrap.cc:654] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:16.091272 6197 ts_tablet_manager.cc:1905] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:16.093547 6195 ts_tablet_manager.cc:1918] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:59:16.093706 6195 log.cc:1196] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/1911c3a2ee1543238d654b35024e54b0
I20260809 00:59:16.094313 6171 tablet_bootstrap.cc:492] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c: Bootstrap starting.
I20260809 00:59:16.094347 6195 ts_tablet_manager.cc:1939] T 1911c3a2ee1543238d654b35024e54b0 P 64d565dacb8346c7bd7f67a6f17a2e5c: Deleting consensus metadata
I20260809 00:59:16.095975 6171 tablet_bootstrap.cc:654] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:16.097422 5603 catalog_manager.cc:4841] TS 64d565dacb8346c7bd7f67a6f17a2e5c (127.4.82.129:37771): tablet 1911c3a2ee1543238d654b35024e54b0 (table test-table [id=0e1ef063235949d38f771ba9118694b9]) successfully deleted
I20260809 00:59:16.099109 6171 tablet_bootstrap.cc:492] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c: No bootstrap required, opened a new log
I20260809 00:59:16.099323 6171 ts_tablet_manager.cc:1397] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 00:59:16.100175 6171 raft_consensus.cc:348] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.100422 6171 raft_consensus.cc:374] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:16.100543 6171 raft_consensus.cc:729] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d565dacb8346c7bd7f67a6f17a2e5c, State: Initialized, Role: FOLLOWER
I20260809 00:59:16.100829 6171 consensus_queue.cc:260] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.101531 6171 ts_tablet_manager.cc:1428] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:59:16.102854 6196 ts_tablet_manager.cc:1918] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:59:16.103045 6196 log.cc:1196] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/1911c3a2ee1543238d654b35024e54b0
I20260809 00:59:16.106096 6196 ts_tablet_manager.cc:1939] T 1911c3a2ee1543238d654b35024e54b0 P 2e90e2a989d64b6d8e426c05657eaa12: Deleting consensus metadata
I20260809 00:59:16.107774 6175 tablet_bootstrap.cc:492] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1: Bootstrap starting.
I20260809 00:59:16.109056 5604 catalog_manager.cc:4841] TS 2e90e2a989d64b6d8e426c05657eaa12 (127.4.82.130:42613): tablet 1911c3a2ee1543238d654b35024e54b0 (table test-table [id=0e1ef063235949d38f771ba9118694b9]) successfully deleted
I20260809 00:59:16.109647 6175 tablet_bootstrap.cc:654] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:16.113461 6197 ts_tablet_manager.cc:1918] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:59:16.114403 6197 log.cc:1196] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/1911c3a2ee1543238d654b35024e54b0
I20260809 00:59:16.114989 6172 tablet_bootstrap.cc:492] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12: No bootstrap required, opened a new log
I20260809 00:59:16.115018 6197 ts_tablet_manager.cc:1939] T 1911c3a2ee1543238d654b35024e54b0 P bf1938eabb154f2eb454acde458b72e1: Deleting consensus metadata
I20260809 00:59:16.115135 6172 ts_tablet_manager.cc:1397] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.012s
I20260809 00:59:16.115147 6175 tablet_bootstrap.cc:492] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1: No bootstrap required, opened a new log
I20260809 00:59:16.115649 6175 ts_tablet_manager.cc:1397] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 00:59:16.116071 6172 raft_consensus.cc:348] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.116304 6172 raft_consensus.cc:374] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:16.116425 6172 raft_consensus.cc:729] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e90e2a989d64b6d8e426c05657eaa12, State: Initialized, Role: FOLLOWER
I20260809 00:59:16.116536 6175 raft_consensus.cc:348] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.116778 6175 raft_consensus.cc:374] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:16.116698 6172 consensus_queue.cc:260] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.116905 6175 raft_consensus.cc:729] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf1938eabb154f2eb454acde458b72e1, State: Initialized, Role: FOLLOWER
I20260809 00:59:16.117352 5604 catalog_manager.cc:4841] TS bf1938eabb154f2eb454acde458b72e1 (127.4.82.131:43661): tablet 1911c3a2ee1543238d654b35024e54b0 (table test-table [id=0e1ef063235949d38f771ba9118694b9]) successfully deleted
I20260809 00:59:16.117424 6172 ts_tablet_manager.cc:1428] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 00:59:16.117333 6175 consensus_queue.cc:260] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.118953 6175 ts_tablet_manager.cc:1428] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:59:16.184665 6180 raft_consensus.cc:482] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:59:16.184873 6180 raft_consensus.cc:504] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.185601 6180 leader_election.cc:288] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64d565dacb8346c7bd7f67a6f17a2e5c (127.4.82.129:37771), 2e90e2a989d64b6d8e426c05657eaa12 (127.4.82.130:42613)
I20260809 00:59:16.186091 5796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3fa0be13f0b4258962f8b4edb7d3415" candidate_uuid: "bf1938eabb154f2eb454acde458b72e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" is_pre_election: true
I20260809 00:59:16.186350 5796 raft_consensus.cc:2454] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf1938eabb154f2eb454acde458b72e1 in term 0.
I20260809 00:59:16.186277 5948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3fa0be13f0b4258962f8b4edb7d3415" candidate_uuid: "bf1938eabb154f2eb454acde458b72e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e90e2a989d64b6d8e426c05657eaa12" is_pre_election: true
I20260809 00:59:16.186524 5948 raft_consensus.cc:2454] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf1938eabb154f2eb454acde458b72e1 in term 0.
I20260809 00:59:16.186934 6018 leader_election.cc:302] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d565dacb8346c7bd7f67a6f17a2e5c, bf1938eabb154f2eb454acde458b72e1; no voters:
I20260809 00:59:16.187454 6180 raft_consensus.cc:2781] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:59:16.187568 6180 raft_consensus.cc:482] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:59:16.187646 6180 raft_consensus.cc:3037] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:16.190754 6180 raft_consensus.cc:504] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.191844 5796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3fa0be13f0b4258962f8b4edb7d3415" candidate_uuid: "bf1938eabb154f2eb454acde458b72e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c"
I20260809 00:59:16.191865 5948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3fa0be13f0b4258962f8b4edb7d3415" candidate_uuid: "bf1938eabb154f2eb454acde458b72e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e90e2a989d64b6d8e426c05657eaa12"
I20260809 00:59:16.192034 5796 raft_consensus.cc:3037] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:16.192062 5948 raft_consensus.cc:3037] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:16.195261 5948 raft_consensus.cc:2454] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf1938eabb154f2eb454acde458b72e1 in term 1.
I20260809 00:59:16.195382 5796 raft_consensus.cc:2454] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf1938eabb154f2eb454acde458b72e1 in term 1.
I20260809 00:59:16.195834 6019 leader_election.cc:302] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e90e2a989d64b6d8e426c05657eaa12, bf1938eabb154f2eb454acde458b72e1; no voters:
I20260809 00:59:16.196041 6180 leader_election.cc:288] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [CANDIDATE]: Term 1 election: Requested vote from peers 64d565dacb8346c7bd7f67a6f17a2e5c (127.4.82.129:37771), 2e90e2a989d64b6d8e426c05657eaa12 (127.4.82.130:42613)
I20260809 00:59:16.196133 6182 raft_consensus.cc:2781] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:16.196465 6182 raft_consensus.cc:686] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [term 1 LEADER]: Becoming Leader. State: Replica: bf1938eabb154f2eb454acde458b72e1, State: Running, Role: LEADER
I20260809 00:59:16.196740 6182 consensus_queue.cc:237] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.199589 5621 catalog_manager.cc:5495] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf1938eabb154f2eb454acde458b72e1 (127.4.82.131). New cstate: current_term: 1 leader_uuid: "bf1938eabb154f2eb454acde458b72e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } health_report { overall_health: HEALTHY } } }
Aug 09 00:59:16 dist-test-slave-07c2 krb5kdc[5571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237153, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:16.260500 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:16.245673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.4.82.190:46197)
0809 00:59:16.246276 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:16.246283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:16.246647 (+ 364us) server_negotiation.cc:407] Connection header received
0809 00:59:16.246810 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:16.246816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:16.246897 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:16.247019 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:16.248232 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.249509 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.250689 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.251390 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.251514 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:16.254969 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:16.254974 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:16.254977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:16.255004 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:16.257622 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:16.258457 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:16.258461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:16.258463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:16.258570 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:16.258976 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:16.258980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:16.258981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:16.259232 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:16.259441 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:16.259941 (+ 500us) server_negotiation.cc:299] Negotiation successful
0809 00:59:16.260207 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":452,"thread_start_us":332,"threads_started":1}
I20260809 00:59:16.260581 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:16.245558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:46197 (local address 127.0.0.1:57628)
0809 00:59:16.246536 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:16.246552 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:16.246674 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:16.247052 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:16.247056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:16.247811 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:16.248064 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:16.248072 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.249634 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.249637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:16.250545 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:16.250555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.251518 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.251522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:16.251670 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:16.252340 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:16.252362 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:16.254759 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:16.257763 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:16.257776 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:16.257784 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:16.258352 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:16.258672 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:16.258677 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:16.258681 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:16.258866 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:16.259414 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:16.259423 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:16.259630 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 00:59:16.259810 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 00:59:16.260086 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":657,"threads_started":1}
I20260809 00:59:16.265698 5621 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:57628:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 00:59:16.266713 5621 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:59:16.273283 5774 tablet_service.cc:1467] Processing CreateTablet for tablet b0ddfcda49de47008027ac5535c8b526 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb86ca405855408bba394c6cfe6b9c9d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:59:16.273885 5774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0ddfcda49de47008027ac5535c8b526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:16.276094 6073 tablet_service.cc:1467] Processing CreateTablet for tablet b0ddfcda49de47008027ac5535c8b526 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb86ca405855408bba394c6cfe6b9c9d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:59:16.276088 5922 tablet_service.cc:1467] Processing CreateTablet for tablet b0ddfcda49de47008027ac5535c8b526 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb86ca405855408bba394c6cfe6b9c9d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:59:16.276654 6073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0ddfcda49de47008027ac5535c8b526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:16.276651 5922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0ddfcda49de47008027ac5535c8b526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:16.281507 6171 tablet_bootstrap.cc:492] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c: Bootstrap starting.
I20260809 00:59:16.283239 6171 tablet_bootstrap.cc:654] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:16.284230 6172 tablet_bootstrap.cc:492] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12: Bootstrap starting.
I20260809 00:59:16.286020 6172 tablet_bootstrap.cc:654] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:16.286993 6175 tablet_bootstrap.cc:492] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1: Bootstrap starting.
I20260809 00:59:16.289364 6175 tablet_bootstrap.cc:654] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:16.295709 6172 tablet_bootstrap.cc:492] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12: No bootstrap required, opened a new log
I20260809 00:59:16.295887 6172 ts_tablet_manager.cc:1397] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260809 00:59:16.296368 6175 tablet_bootstrap.cc:492] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1: No bootstrap required, opened a new log
I20260809 00:59:16.296550 6175 ts_tablet_manager.cc:1397] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260809 00:59:16.296691 6172 raft_consensus.cc:348] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.296943 6172 raft_consensus.cc:374] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:16.297027 6172 raft_consensus.cc:729] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e90e2a989d64b6d8e426c05657eaa12, State: Initialized, Role: FOLLOWER
I20260809 00:59:16.297261 6172 consensus_queue.cc:260] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.297361 6175 raft_consensus.cc:348] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.297573 6175 raft_consensus.cc:374] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:16.297648 6175 raft_consensus.cc:729] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf1938eabb154f2eb454acde458b72e1, State: Initialized, Role: FOLLOWER
I20260809 00:59:16.297966 6172 ts_tablet_manager.cc:1428] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:59:16.297845 6177 tablet_replica.cc:439] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } }
I20260809 00:59:16.298493 6175 consensus_queue.cc:260] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.299170 6175 ts_tablet_manager.cc:1428] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:59:16.299511 6177 tablet_replica.cc:442] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:16.301600 6182 tablet_replica.cc:439] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } }
I20260809 00:59:16.301908 6182 tablet_replica.cc:442] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:16.303260 6171 tablet_bootstrap.cc:492] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c: No bootstrap required, opened a new log
I20260809 00:59:16.303462 6171 ts_tablet_manager.cc:1397] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.010s
I20260809 00:59:16.304167 6171 raft_consensus.cc:348] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.304402 6171 raft_consensus.cc:374] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:16.304484 6171 raft_consensus.cc:729] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d565dacb8346c7bd7f67a6f17a2e5c, State: Initialized, Role: FOLLOWER
I20260809 00:59:16.304716 6171 consensus_queue.cc:260] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.305374 6171 ts_tablet_manager.cc:1428] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:59:16.305269 6179 tablet_replica.cc:439] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } }
I20260809 00:59:16.305651 6179 tablet_replica.cc:442] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:16.542997 6182 raft_consensus.cc:482] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:59:16.543228 6182 raft_consensus.cc:504] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.543968 6182 leader_election.cc:288] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64d565dacb8346c7bd7f67a6f17a2e5c (127.4.82.129:37771), 2e90e2a989d64b6d8e426c05657eaa12 (127.4.82.130:42613)
I20260809 00:59:16.544548 5796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ddfcda49de47008027ac5535c8b526" candidate_uuid: "bf1938eabb154f2eb454acde458b72e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" is_pre_election: true
I20260809 00:59:16.544559 5948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ddfcda49de47008027ac5535c8b526" candidate_uuid: "bf1938eabb154f2eb454acde458b72e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e90e2a989d64b6d8e426c05657eaa12" is_pre_election: true
I20260809 00:59:16.544823 5948 raft_consensus.cc:2454] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf1938eabb154f2eb454acde458b72e1 in term 0.
I20260809 00:59:16.544842 5796 raft_consensus.cc:2454] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf1938eabb154f2eb454acde458b72e1 in term 0.
I20260809 00:59:16.545288 6019 leader_election.cc:302] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e90e2a989d64b6d8e426c05657eaa12, bf1938eabb154f2eb454acde458b72e1; no voters:
I20260809 00:59:16.545565 6182 raft_consensus.cc:2781] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:59:16.545696 6182 raft_consensus.cc:482] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:59:16.545771 6182 raft_consensus.cc:3037] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:16.547758 6182 raft_consensus.cc:504] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.548242 6182 leader_election.cc:288] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [CANDIDATE]: Term 1 election: Requested vote from peers 64d565dacb8346c7bd7f67a6f17a2e5c (127.4.82.129:37771), 2e90e2a989d64b6d8e426c05657eaa12 (127.4.82.130:42613)
I20260809 00:59:16.548604 5796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ddfcda49de47008027ac5535c8b526" candidate_uuid: "bf1938eabb154f2eb454acde458b72e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c"
I20260809 00:59:16.548728 5948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ddfcda49de47008027ac5535c8b526" candidate_uuid: "bf1938eabb154f2eb454acde458b72e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e90e2a989d64b6d8e426c05657eaa12"
I20260809 00:59:16.548834 5796 raft_consensus.cc:3037] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:16.548993 5948 raft_consensus.cc:3037] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:16.550976 5796 raft_consensus.cc:2454] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf1938eabb154f2eb454acde458b72e1 in term 1.
I20260809 00:59:16.551189 5948 raft_consensus.cc:2454] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf1938eabb154f2eb454acde458b72e1 in term 1.
I20260809 00:59:16.551450 6018 leader_election.cc:302] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d565dacb8346c7bd7f67a6f17a2e5c, bf1938eabb154f2eb454acde458b72e1; no voters:
I20260809 00:59:16.551767 6182 raft_consensus.cc:2781] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:16.551947 6182 raft_consensus.cc:686] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [term 1 LEADER]: Becoming Leader. State: Replica: bf1938eabb154f2eb454acde458b72e1, State: Running, Role: LEADER
I20260809 00:59:16.552275 6182 consensus_queue.cc:237] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } }
I20260809 00:59:16.553876 6180 tablet_replica.cc:439] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1: TxnStatusTablet state changed. Reason: New leader bf1938eabb154f2eb454acde458b72e1. Latest consensus state: current_term: 1 leader_uuid: "bf1938eabb154f2eb454acde458b72e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } }
I20260809 00:59:16.554108 6180 tablet_replica.cc:442] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:59:16.554826 6214 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 00:59:16.555135 6214 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 00:59:16.555713 5621 catalog_manager.cc:5495] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf1938eabb154f2eb454acde458b72e1 (127.4.82.131). New cstate: current_term: 1 leader_uuid: "bf1938eabb154f2eb454acde458b72e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } health_report { overall_health: HEALTHY } } }
I20260809 00:59:16.592584 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:16.583589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:43661 (local address 127.0.0.1:49630)
0809 00:59:16.583850 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:16.583870 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:16.583983 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:16.588076 (+ 4093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:16.588080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:16.588093 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:59:16.588292 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:16.588301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.589462 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.589468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:16.590506 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:16.590518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.591326 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.591333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:16.591497 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:16.591557 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:59:16.591997 (+ 440us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:59:16.592023 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 00:59:16.592123 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 00:59:16.592283 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 00:59:16.592614 6216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:16.583636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49630 (local address 127.4.82.131:43661)
0809 00:59:16.587760 (+ 4124us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:16.587764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:16.587781 (+ 17us) server_negotiation.cc:407] Connection header received
0809 00:59:16.587839 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:16.587842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:16.587913 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:16.588044 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:59:16.588467 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.589314 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.590664 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.591137 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.591284 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:16.591754 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:59:16.591886 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:59:16.591996 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:16.592154 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 00:59:16.592231 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3972,"thread_start_us":3823,"threads_started":1}
I20260809 00:59:16.597601 5948 raft_consensus.cc:1261] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12 [term 1 FOLLOWER]: Refusing update from remote peer bf1938eabb154f2eb454acde458b72e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:16.597673 5796 raft_consensus.cc:1261] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 1 FOLLOWER]: Refusing update from remote peer bf1938eabb154f2eb454acde458b72e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:16.598299 6180 consensus_queue.cc:1035] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:16.598601 6182 consensus_queue.cc:1035] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:16.608098 6179 tablet_replica.cc:439] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c: TxnStatusTablet state changed. Reason: New leader bf1938eabb154f2eb454acde458b72e1. Latest consensus state: current_term: 1 leader_uuid: "bf1938eabb154f2eb454acde458b72e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } }
I20260809 00:59:16.608373 6179 tablet_replica.cc:442] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:16.612187 6177 tablet_replica.cc:439] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12: TxnStatusTablet state changed. Reason: New leader bf1938eabb154f2eb454acde458b72e1. Latest consensus state: current_term: 1 leader_uuid: "bf1938eabb154f2eb454acde458b72e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } }
I20260809 00:59:16.612483 6177 tablet_replica.cc:442] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:16.612831 6182 tablet_replica.cc:439] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf1938eabb154f2eb454acde458b72e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } }
I20260809 00:59:16.613175 6182 tablet_replica.cc:442] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:59:16.617341 6179 tablet_replica.cc:439] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf1938eabb154f2eb454acde458b72e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } }
I20260809 00:59:16.617616 6179 tablet_replica.cc:442] T b0ddfcda49de47008027ac5535c8b526 P 64d565dacb8346c7bd7f67a6f17a2e5c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:16.624105 6180 tablet_replica.cc:439] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bf1938eabb154f2eb454acde458b72e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } }
I20260809 00:59:16.624373 6180 tablet_replica.cc:442] T b0ddfcda49de47008027ac5535c8b526 P bf1938eabb154f2eb454acde458b72e1: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:59:16.628377 6177 tablet_replica.cc:439] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bf1938eabb154f2eb454acde458b72e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 } } peers { permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 } } peers { permanent_uuid: "bf1938eabb154f2eb454acde458b72e1" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 43661 } } }
I20260809 00:59:16.628674 6177 tablet_replica.cc:442] T b0ddfcda49de47008027ac5535c8b526 P 2e90e2a989d64b6d8e426c05657eaa12: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:16.645255 6216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:16.638832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:44775 (local address 127.4.82.131:43661)
0809 00:59:16.639012 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:16.639019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:16.639638 (+ 619us) server_negotiation.cc:407] Connection header received
0809 00:59:16.639797 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:16.639803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:16.639884 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:16.639985 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:59:16.640577 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.641813 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.643266 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.643976 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.644100 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:16.644463 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:59:16.644600 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:59:16.644695 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:16.644801 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 00:59:16.644916 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":56}
I20260809 00:59:16.645254 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:16.638840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:43661 (local address 127.4.82.131:44775)
0809 00:59:16.639543 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:16.639564 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:16.639675 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:16.640006 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:16.640008 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:16.640017 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:59:16.640393 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:16.640403 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.641940 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.641944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:16.643107 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:16.643118 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.644137 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.644141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:16.644256 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:16.644308 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:59:16.644699 (+ 391us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:59:16.644708 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 00:59:16.644778 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:59:16.644890 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":1,"thread_start_us":375,"threads_started":1}
I20260809 00:59:16.654706 5948 raft_consensus.cc:1261] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12 [term 1 FOLLOWER]: Refusing update from remote peer bf1938eabb154f2eb454acde458b72e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:16.654803 5796 raft_consensus.cc:1261] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 1 FOLLOWER]: Refusing update from remote peer bf1938eabb154f2eb454acde458b72e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:16.655397 6182 consensus_queue.cc:1035] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64d565dacb8346c7bd7f67a6f17a2e5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:16.655850 6180 consensus_queue.cc:1035] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e90e2a989d64b6d8e426c05657eaa12" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:16.674791 6219 mvcc.cc:205] Tried to move back new op lower bound from 7316427393652203520 to 7316427391784361984. Current Snapshot: MvccSnapshot[applied={T|T < 7316427393652203520}]
I20260809 00:59:16.746891 5621 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:16.747107 5621 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:16.749871 5621 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4334c06dc380492487ad62b7cdd5a2cf: Sending DeleteTablet for 3 replicas of tablet d3fa0be13f0b4258962f8b4edb7d3415
I20260809 00:59:16.750797 5774 tablet_service.cc:1514] Processing DeleteTablet for tablet d3fa0be13f0b4258962f8b4edb7d3415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:16 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:43866
I20260809 00:59:16.750991 6072 tablet_service.cc:1514] Processing DeleteTablet for tablet d3fa0be13f0b4258962f8b4edb7d3415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:16 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:49608
I20260809 00:59:16.751228 5922 tablet_service.cc:1514] Processing DeleteTablet for tablet d3fa0be13f0b4258962f8b4edb7d3415 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:16 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:35812
I20260809 00:59:16.764796 6234 tablet_replica.cc:330] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1: stopping tablet replica
I20260809 00:59:16.765081 6234 raft_consensus.cc:2229] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:59:16.765976 6234 raft_consensus.cc:2258] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:16.767159 6234 ts_tablet_manager.cc:1905] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:16.773895 6234 ts_tablet_manager.cc:1918] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:59:16.774050 6234 log.cc:1196] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/d3fa0be13f0b4258962f8b4edb7d3415
I20260809 00:59:16.774678 6234 ts_tablet_manager.cc:1939] T d3fa0be13f0b4258962f8b4edb7d3415 P bf1938eabb154f2eb454acde458b72e1: Deleting consensus metadata
I20260809 00:59:16.776083 5604 catalog_manager.cc:4841] TS bf1938eabb154f2eb454acde458b72e1 (127.4.82.131:43661): tablet d3fa0be13f0b4258962f8b4edb7d3415 (table test-table [id=d5736664ec254d1986f45a89838ff428]) successfully deleted
I20260809 00:59:16.777539 6235 tablet_replica.cc:330] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12: stopping tablet replica
I20260809 00:59:16.777743 6235 raft_consensus.cc:2229] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:16.777994 6235 raft_consensus.cc:2258] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:16.778637 6233 tablet_replica.cc:330] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c: stopping tablet replica
I20260809 00:59:16.778841 6233 raft_consensus.cc:2229] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:16.779922 6235 ts_tablet_manager.cc:1905] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:16.780668 6233 raft_consensus.cc:2258] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:16.787034 6235 ts_tablet_manager.cc:1918] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:59:16.787177 6235 log.cc:1196] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/d3fa0be13f0b4258962f8b4edb7d3415
I20260809 00:59:16.787676 6235 ts_tablet_manager.cc:1939] T d3fa0be13f0b4258962f8b4edb7d3415 P 2e90e2a989d64b6d8e426c05657eaa12: Deleting consensus metadata
I20260809 00:59:16.788705 5604 catalog_manager.cc:4841] TS 2e90e2a989d64b6d8e426c05657eaa12 (127.4.82.130:42613): tablet d3fa0be13f0b4258962f8b4edb7d3415 (table test-table [id=d5736664ec254d1986f45a89838ff428]) successfully deleted
I20260809 00:59:16.788764 6233 ts_tablet_manager.cc:1905] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:16.796053 6233 ts_tablet_manager.cc:1918] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:59:16.796191 6233 log.cc:1196] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/d3fa0be13f0b4258962f8b4edb7d3415
I20260809 00:59:16.796727 6233 ts_tablet_manager.cc:1939] T d3fa0be13f0b4258962f8b4edb7d3415 P 64d565dacb8346c7bd7f67a6f17a2e5c: Deleting consensus metadata
I20260809 00:59:16.797871 5603 catalog_manager.cc:4841] TS 64d565dacb8346c7bd7f67a6f17a2e5c (127.4.82.129:37771): tablet d3fa0be13f0b4258962f8b4edb7d3415 (table test-table [id=d5736664ec254d1986f45a89838ff428]) successfully deleted
Aug 09 00:59:16 dist-test-slave-07c2 krb5kdc[5571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237155, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 00:59:16.806481 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:16.771525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.4.82.129:37771)
0809 00:59:16.789081 (+ 17556us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:16.789086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:16.789107 (+ 21us) server_negotiation.cc:407] Connection header received
0809 00:59:16.789155 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:16.789159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:16.789217 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:16.789345 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:16.791363 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.792670 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.794633 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.795363 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.795495 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:16.800635 (+ 5140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:16.800647 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:16.800663 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:16.800698 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:16.803673 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:16.804493 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:16.804500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:16.804502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:16.804642 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:16.805029 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:16.805032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:16.805034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:16.805396 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:16.805660 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:16.805989 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 00:59:16.806170 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17440,"thread_start_us":17340,"threads_started":1}
W20260809 00:59:16.807144 5842 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43870
I20260809 00:59:16.824120 6240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:16.810173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57642 (local address 127.4.82.190:46197)
0809 00:59:16.810646 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:16.810655 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:16.811127 (+ 472us) server_negotiation.cc:407] Connection header received
0809 00:59:16.811280 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:16.811342 (+ 62us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:16.811412 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:16.811498 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:16.813078 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.813827 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.815609 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.816026 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.816118 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:16.818959 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:16.818964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:16.818967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:16.818998 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:16.821568 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:16.822301 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:16.822304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:16.822306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:16.822427 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:16.822791 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:16.822794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:16.822795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:16.823042 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:16.823227 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:16.823639 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 00:59:16.823802 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":229,"threads_started":1}
W20260809 00:59:16.824689 5621 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57642
I20260809 00:59:16.826553 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 5701
I20260809 00:59:16.840777 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:16.827440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43880 (local address 127.4.82.129:37771)
0809 00:59:16.827608 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:16.827613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:16.828536 (+ 923us) server_negotiation.cc:407] Connection header received
0809 00:59:16.828666 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:16.828672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:16.828784 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:16.828889 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:16.829916 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.830987 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.832208 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.832849 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.832986 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:16.835023 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:16.835029 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:16.835033 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:16.835079 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:16.838281 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:16.838886 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:16.838891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:16.838895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:16.839063 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:16.839430 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:16.839437 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:16.839440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:16.839731 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:16.839959 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:16.840260 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 00:59:16.840461 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 00:59:16.841369 5842 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43880
W20260809 00:59:16.841929 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 5701: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 00:59:16.842113 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 5701
I20260809 00:59:16.868588 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 5851
Aug 09 00:59:16 dist-test-slave-07c2 krb5kdc[5571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237155, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 00:59:16.885396 6243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:16.869395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35816 (local address 127.4.82.130:42613)
0809 00:59:16.869894 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:16.869898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:16.869912 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:59:16.869964 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:16.869968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:16.870026 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:16.870147 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:16.871385 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.872094 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.873327 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.873828 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.874040 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:16.878338 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:16.878362 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:16.878382 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:16.878428 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:16.882540 (+ 4112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:16.883191 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:16.883203 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:16.883207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:16.883433 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:16.883785 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:16.883791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:16.883794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:16.884265 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:16.884576 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:16.884723 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 00:59:16.884970 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":268,"threads_started":1}
W20260809 00:59:16.886152 5989 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35816
W20260809 00:59:16.886817 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 5851: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 00:59:16.886986 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 5851
I20260809 00:59:16.910447 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 6000
Aug 09 00:59:16 dist-test-slave-07c2 krb5kdc[5571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237155, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 00:59:16.927927 6216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:16.911288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.4.82.131:43661)
0809 00:59:16.911534 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:16.911540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:16.911559 (+ 19us) server_negotiation.cc:407] Connection header received
0809 00:59:16.911767 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:16.911772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:16.911868 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:16.911985 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:16.913245 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.914045 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.915281 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:16.915810 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:16.915943 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:16.920643 (+ 4700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:16.920663 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:16.920685 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:16.920730 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:16.924896 (+ 4166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:16.925558 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:16.925568 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:16.925572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:16.925758 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:16.926125 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:16.926130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:16.926133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:16.926648 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:16.926973 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:16.927202 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 00:59:16.927486 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 00:59:16.928690 6145 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49638
W20260809 00:59:16.929409 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 6000: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 00:59:16.929535 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 6000
I20260809 00:59:16.956405 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 5584
I20260809 00:59:16.957477 5665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:17.073031 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 5584
2026-08-09T00:59:17Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4742 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:59:17 dist-test-slave-07c2 krb5kdc[6247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:59:17 dist-test-slave-07c2 krb5kdc[6247](info): set up 2 sockets
Aug 09 00:59:17 dist-test-slave-07c2 krb5kdc[6247](info): commencing operation
krb5kdc: starting...
W20260809 00:59:18.251443 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 00:59:18 dist-test-slave-07c2 krb5kdc[6247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:59:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 00:59:18.440026 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:32915
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:41735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:32915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:18.752719 6262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:18.752990 6262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:18.753188 6262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:18.760267 6262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:18.760361 6262 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 00:59:18.760452 6262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:18.760555 6262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:18.760653 6262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:18.760763 6262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:18.770287 6262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:41735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:32915
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:32915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:18.771495 6262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:18.772764 6262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:18.783897 6268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:18.783941 6270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:18.784201 6267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:18.784922 6262 server_base.cc:1034] running on GCE node
I20260809 00:59:18.785604 6262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:18.786854 6262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:18.788091 6262 hybrid_clock.cc:648] HybridClock initialized: now 1786237158788039 us; error 29 us; skew 500 ppm
I20260809 00:59:18.788403 6262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:18 dist-test-slave-07c2 krb5kdc[6247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237158, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:18.793874 6262 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 00:59:18.795225 6262 webserver.cc:460] Webserver started at http://127.4.82.190:43789/ using document root <none> and password file <none>
I20260809 00:59:18.795912 6262 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:18.796044 6262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:18.796355 6262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:18.799149 6262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "9f38c035462e47f1b87fc53b68e5a4d6"
format_stamp: "Formatted at 2026-08-09 00:59:18 on dist-test-slave-07c2"
server_key: "2fac47c38f174b8979caca47775f0a82"
server_key_iv: "e08287bfa139fe8c855036ea0809529b"
server_key_version: "encryptionkey@0"
I20260809 00:59:18.799961 6262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "9f38c035462e47f1b87fc53b68e5a4d6"
format_stamp: "Formatted at 2026-08-09 00:59:18 on dist-test-slave-07c2"
server_key: "2fac47c38f174b8979caca47775f0a82"
server_key_iv: "e08287bfa139fe8c855036ea0809529b"
server_key_version: "encryptionkey@0"
I20260809 00:59:18.804773 6262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 00:59:18.807938 6278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:18.809156 6262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 00:59:18.809327 6262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "9f38c035462e47f1b87fc53b68e5a4d6"
format_stamp: "Formatted at 2026-08-09 00:59:18 on dist-test-slave-07c2"
server_key: "2fac47c38f174b8979caca47775f0a82"
server_key_iv: "e08287bfa139fe8c855036ea0809529b"
server_key_version: "encryptionkey@0"
I20260809 00:59:18.809533 6262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:18.819651 6262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:18.823165 6262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:18.823474 6262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:18.851547 6262 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:32915
I20260809 00:59:18.851572 6345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:32915 every 8 connection(s)
I20260809 00:59:18.853555 6262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:18.856267 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 6262
I20260809 00:59:18.856491 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:18.857409 4426 external_mini_cluster.cc:1428] Setting key 05866de9a53d61a353e0e06d5d7520a8
I20260809 00:59:18.858778 6346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:18.874089 6346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6: Bootstrap starting.
I20260809 00:59:18.877758 6346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:18.879025 6346 log.cc:824] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6: Log is configured to *not* fsync() on all Append() calls
Aug 09 00:59:18 dist-test-slave-07c2 krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:18.882576 6346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6: No bootstrap required, opened a new log
I20260809 00:59:18.888983 6349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:18.859775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34196 (local address 127.4.82.190:32915)
0809 00:59:18.860726 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:18.860743 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:18.860796 (+ 53us) server_negotiation.cc:407] Connection header received
0809 00:59:18.863498 (+ 2702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:18.863529 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:18.864066 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:18.864556 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:18.866286 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:18.868687 (+ 2401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:18.875438 (+ 6751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:18.876689 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:18.876892 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:18.881913 (+ 5021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:18.881923 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:18.881940 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:18.881982 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:18.885054 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:18.885900 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:18.885905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:18.885911 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:18.886065 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:18.886475 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:18.886479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:18.886481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:18.886938 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:18.887163 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:18.887637 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 00:59:18.887959 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":433,"threads_started":1}
I20260809 00:59:18.891215 6346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f38c035462e47f1b87fc53b68e5a4d6" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 32915 } }
I20260809 00:59:18.891625 6346 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:18.891745 6346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f38c035462e47f1b87fc53b68e5a4d6, State: Initialized, Role: FOLLOWER
I20260809 00:59:18.892251 6346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f38c035462e47f1b87fc53b68e5a4d6" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 32915 } }
I20260809 00:59:18.892483 6346 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:18.892625 6346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:18.892795 6346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:18.894804 6346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f38c035462e47f1b87fc53b68e5a4d6" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 32915 } }
I20260809 00:59:18.895278 6346 leader_election.cc:302] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f38c035462e47f1b87fc53b68e5a4d6; no voters:
I20260809 00:59:18.895983 6346 leader_election.cc:288] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:18.896093 6351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:18.896735 6351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [term 1 LEADER]: Becoming Leader. State: Replica: 9f38c035462e47f1b87fc53b68e5a4d6, State: Running, Role: LEADER
I20260809 00:59:18.897261 6351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f38c035462e47f1b87fc53b68e5a4d6" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 32915 } }
I20260809 00:59:18.898084 6346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:18.900287 6353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f38c035462e47f1b87fc53b68e5a4d6. Latest consensus state: current_term: 1 leader_uuid: "9f38c035462e47f1b87fc53b68e5a4d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f38c035462e47f1b87fc53b68e5a4d6" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 32915 } } }
I20260809 00:59:18.900357 6352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f38c035462e47f1b87fc53b68e5a4d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f38c035462e47f1b87fc53b68e5a4d6" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 32915 } } }
I20260809 00:59:18.900686 6353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:18.900724 6352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:18.902508 6358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:18.907804 6358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:18.916492 6358 catalog_manager.cc:1347] Generated new cluster ID: 5b10516c2b9949bbba2c5377d83f9bfe
I20260809 00:59:18.916613 6358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:18.931006 6358 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:18.932304 6358 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:18.944968 6358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6: Generated new TSK 0
I20260809 00:59:18.945911 6358 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 00:59:19.000437 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:32915
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:41735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:19.323450 6374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:19.323747 6374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:19.323921 6374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:19.331198 6374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:19.331378 6374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:19.331524 6374 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 00:59:19.331614 6374 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:59:19.331755 6374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 00:59:19.341300 6374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:41735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.82.190:32915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:19.342598 6374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:19.344017 6374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:19.356441 6379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:19.356550 6382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:19.356472 6380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:19.357447 6374 server_base.cc:1034] running on GCE node
I20260809 00:59:19.358012 6374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:19.358898 6374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:19.360116 6374 hybrid_clock.cc:648] HybridClock initialized: now 1786237159360078 us; error 57 us; skew 500 ppm
I20260809 00:59:19.360421 6374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:19 dist-test-slave-07c2 krb5kdc[6247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237159, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:19.363494 6374 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 00:59:19.364799 6374 webserver.cc:460] Webserver started at http://127.4.82.129:40403/ using document root <none> and password file <none>
I20260809 00:59:19.365449 6374 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:19.365593 6374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:19.365900 6374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:19.368913 6374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "7aa0479418f046048b3cff9079fa4a21"
format_stamp: "Formatted at 2026-08-09 00:59:19 on dist-test-slave-07c2"
server_key: "d6814d43ca97b0da9f4252227c45401c"
server_key_iv: "de2914b9b83f36147e78dc8c163c0cbb"
server_key_version: "encryptionkey@0"
I20260809 00:59:19.369697 6374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "7aa0479418f046048b3cff9079fa4a21"
format_stamp: "Formatted at 2026-08-09 00:59:19 on dist-test-slave-07c2"
server_key: "d6814d43ca97b0da9f4252227c45401c"
server_key_iv: "de2914b9b83f36147e78dc8c163c0cbb"
server_key_version: "encryptionkey@0"
I20260809 00:59:19.374651 6374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 00:59:19.377998 6390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:19.379264 6374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 00:59:19.379458 6374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "7aa0479418f046048b3cff9079fa4a21"
format_stamp: "Formatted at 2026-08-09 00:59:19 on dist-test-slave-07c2"
server_key: "d6814d43ca97b0da9f4252227c45401c"
server_key_iv: "de2914b9b83f36147e78dc8c163c0cbb"
server_key_version: "encryptionkey@0"
I20260809 00:59:19.379676 6374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:19.394292 6374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:19.397982 6374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:19.398355 6374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:19.401021 6374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:19.401158 6374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:19.401281 6374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:19.401386 6374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:59:19 dist-test-slave-07c2 krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237159, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:19.425395 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:19.401289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:43997 (local address 127.4.82.190:32915)
0809 00:59:19.403540 (+ 2251us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:19.403547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:19.403570 (+ 23us) server_negotiation.cc:407] Connection header received
0809 00:59:19.405053 (+ 1483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:19.405060 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:19.405142 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:19.405271 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:19.407124 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.408141 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.411469 (+ 3328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.411890 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.411990 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:19.417682 (+ 5692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:19.417686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:19.417690 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:19.417723 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:19.421690 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:19.422422 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:19.422430 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:19.422437 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:19.422619 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:19.423044 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:19.423060 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:19.423066 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:19.423511 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:19.424277 (+ 766us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:19.424791 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 00:59:19.425011 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2123,"thread_start_us":2013,"threads_started":1}
I20260809 00:59:19.426757 6400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:19.401837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:32915 (local address 127.4.82.129:43997)
0809 00:59:19.402772 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:19.402821 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:19.404783 (+ 1962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:19.405501 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:19.405508 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:19.406131 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:19.406954 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:19.406976 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.408339 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.408350 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:19.411182 (+ 2832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:19.411213 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.412157 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.412167 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:19.412545 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:19.413452 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:19.413484 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:19.417451 (+ 3967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:19.421837 (+ 4386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:19.421846 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:19.421859 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:19.422284 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:19.422741 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:19.422744 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:19.422747 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:19.422924 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:19.423699 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:19.423715 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:19.424306 (+ 591us) client_negotiation.cc:769] Sending connection context
0809 00:59:19.424694 (+ 388us) client_negotiation.cc:240] Negotiation successful
0809 00:59:19.425222 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":416,"threads_started":1}
I20260809 00:59:19.449821 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:19.433911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:48577 (local address 127.4.82.190:32915)
0809 00:59:19.434342 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:19.434346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:19.435146 (+ 800us) server_negotiation.cc:407] Connection header received
0809 00:59:19.435367 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:19.435374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:19.435463 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:19.435572 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:19.436960 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.437622 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.440169 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.440960 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.441075 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:19.443805 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:19.443812 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:19.443818 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:19.443857 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:19.447276 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:19.447921 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:19.447926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:19.447930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:19.448063 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:19.448428 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:19.448433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:19.448437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:19.448711 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:19.448866 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:19.449275 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 00:59:19.449486 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329}
I20260809 00:59:19.449936 6483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:19.433995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:32915 (local address 127.4.82.129:48577)
0809 00:59:19.435018 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:19.435043 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:19.435212 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:19.435632 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:19.435639 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:19.436365 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:19.436812 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:19.436828 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.437760 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.437768 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:19.439943 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:19.439963 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.441368 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.441376 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:19.441582 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:19.442195 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:19.442228 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:19.443599 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:19.447407 (+ 3808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:19.447413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:19.447419 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:19.447797 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:19.448174 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:19.448180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:19.448183 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:19.448314 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:19.448830 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:19.448835 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:19.449007 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 00:59:19.449159 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 00:59:19.449468 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"thread_start_us":640,"threads_started":1}
I20260809 00:59:19.457290 6374 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:36667
I20260809 00:59:19.457332 6518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:36667 every 8 connection(s)
I20260809 00:59:19.458422 6374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 00:59:19.459704 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 6374
I20260809 00:59:19.459925 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 00:59:19.460775 4426 external_mini_cluster.cc:1428] Setting key fcab6769e0bd9af0b5687808566f6a36
I20260809 00:59:19.461311 6519 heartbeater.cc:347] Connected to a master server at 127.4.82.190:32915
I20260809 00:59:19.461544 6519 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:19.462220 6519 heartbeater.cc:511] Master 127.4.82.190:32915 requested a full tablet report, sending...
I20260809 00:59:19.464342 6296 ts_manager.cc:194] Registered new tserver with Master: 7aa0479418f046048b3cff9079fa4a21 (127.4.82.129:36667)
I20260809 00:59:19.466373 6296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:43997
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 00:59:19.503357 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:32915
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:41735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:19.823885 6524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:19.824201 6524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:19.824388 6524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:19.831835 6524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:19.831975 6524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:19.832131 6524 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 00:59:19.832221 6524 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:59:19.832358 6524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 00:59:19.841850 6524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:41735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.82.190:32915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:19.843158 6524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:19.844472 6524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:19.857479 6530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:19.857579 6531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:19.860400 6534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:19.861039 6524 server_base.cc:1034] running on GCE node
I20260809 00:59:19.861694 6524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:19.862804 6524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:19.864038 6524 hybrid_clock.cc:648] HybridClock initialized: now 1786237159863961 us; error 87 us; skew 500 ppm
I20260809 00:59:19.864400 6524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:19 dist-test-slave-07c2 krb5kdc[6247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237159, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:19.868109 6524 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 00:59:19.870891 6524 webserver.cc:460] Webserver started at http://127.4.82.130:37379/ using document root <none> and password file <none>
I20260809 00:59:19.871639 6524 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:19.871794 6524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:19.872108 6524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:19.874958 6524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "1f5d7a319df74a57888a16783b74df84"
format_stamp: "Formatted at 2026-08-09 00:59:19 on dist-test-slave-07c2"
server_key: "29866945acc5761223c7dcd2c5b6825a"
server_key_iv: "d3af8c7ca74741bd64a851deea27483d"
server_key_version: "encryptionkey@0"
I20260809 00:59:19.875732 6524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "1f5d7a319df74a57888a16783b74df84"
format_stamp: "Formatted at 2026-08-09 00:59:19 on dist-test-slave-07c2"
server_key: "29866945acc5761223c7dcd2c5b6825a"
server_key_iv: "d3af8c7ca74741bd64a851deea27483d"
server_key_version: "encryptionkey@0"
I20260809 00:59:19.880761 6524 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 00:59:19.884299 6542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:19.885535 6524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 00:59:19.885722 6524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "1f5d7a319df74a57888a16783b74df84"
format_stamp: "Formatted at 2026-08-09 00:59:19 on dist-test-slave-07c2"
server_key: "29866945acc5761223c7dcd2c5b6825a"
server_key_iv: "d3af8c7ca74741bd64a851deea27483d"
server_key_version: "encryptionkey@0"
I20260809 00:59:19.885917 6524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:19.922972 6524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:19.927451 6524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:19.927768 6524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:19.930675 6524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:19.930823 6524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:19.930960 6524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:19.931057 6524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:59:19 dist-test-slave-07c2 krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237159, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:19.953508 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:19.931495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:34277 (local address 127.4.82.190:32915)
0809 00:59:19.931703 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:19.931708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:19.933156 (+ 1448us) server_negotiation.cc:407] Connection header received
0809 00:59:19.935200 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:19.935207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:19.935343 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:19.935455 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:19.938363 (+ 2908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.939443 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.941044 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.941535 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.941689 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:19.946339 (+ 4650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:19.946348 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:19.946357 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:19.946400 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:19.949922 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:19.950785 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:19.950794 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:19.950800 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:19.951012 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:19.951561 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:19.951566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:19.951570 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:19.951912 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:19.952039 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:19.953096 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0809 00:59:19.953264 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 00:59:19.955543 6552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:19.932033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:32915 (local address 127.4.82.130:34277)
0809 00:59:19.932968 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:19.933014 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:19.934889 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:19.935925 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:19.935942 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:19.936922 (+ 980us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:19.938116 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:19.938154 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.939617 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.939622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:19.940898 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:19.940911 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.941800 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.941809 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:19.942031 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:19.942728 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:19.942756 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:19.946036 (+ 3280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:19.950084 (+ 4048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:19.950090 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:19.950100 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:19.950652 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:19.951141 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:19.951144 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:19.951146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:19.951408 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:19.952098 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:19.952110 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:19.952656 (+ 546us) client_negotiation.cc:769] Sending connection context
0809 00:59:19.953017 (+ 361us) client_negotiation.cc:240] Negotiation successful
0809 00:59:19.953672 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":448,"threads_started":1}
I20260809 00:59:19.979338 6524 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:37683
I20260809 00:59:19.979360 6675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:37683 every 8 connection(s)
I20260809 00:59:19.980932 6524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 00:59:19.982792 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 6524
I20260809 00:59:19.983008 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 00:59:19.983364 6676 heartbeater.cc:347] Connected to a master server at 127.4.82.190:32915
I20260809 00:59:19.983667 6676 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:19.983743 4426 external_mini_cluster.cc:1428] Setting key 03ac436f86ef5c3809edf6f8ef9ca870
I20260809 00:59:19.983865 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:19.963113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:34157 (local address 127.4.82.190:32915)
0809 00:59:19.963343 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:19.963351 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:19.967995 (+ 4644us) server_negotiation.cc:407] Connection header received
0809 00:59:19.968304 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:19.968312 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:19.968399 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:19.968500 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:19.970012 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.970925 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.973008 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.973647 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.973813 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:19.976767 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:19.976775 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:19.976781 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:19.976826 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:19.980528 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:19.981254 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:19.981261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:19.981265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:19.981454 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:19.981967 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:19.981972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:19.981975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:19.982454 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:19.982583 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:19.983220 (+ 637us) server_negotiation.cc:299] Negotiation successful
0809 00:59:19.983485 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 00:59:19.984936 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:19.962663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:32915 (local address 127.4.82.130:34157)
0809 00:59:19.967855 (+ 5192us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:19.967887 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:19.968065 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:19.968527 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:19.968532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:19.969301 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:19.969864 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:19.969882 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.971053 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.971058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:19.972857 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:19.972873 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:19.973874 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:19.973883 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:19.974142 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:19.974911 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:19.974937 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:19.976459 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:19.980653 (+ 4194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:19.980657 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:19.980659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:19.981117 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:19.981617 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:19.981623 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:19.981626 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:19.981813 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:19.982661 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:19.982665 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:19.982940 (+ 275us) client_negotiation.cc:769] Sending connection context
0809 00:59:19.984077 (+ 1137us) client_negotiation.cc:240] Negotiation successful
0809 00:59:19.984440 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4995,"thread_start_us":4832,"threads_started":1}
I20260809 00:59:19.986017 6676 heartbeater.cc:511] Master 127.4.82.190:32915 requested a full tablet report, sending...
I20260809 00:59:19.987418 6296 ts_manager.cc:194] Registered new tserver with Master: 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683)
I20260809 00:59:19.988220 6296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:34277
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 00:59:20.028424 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:32915
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:41735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:20.347636 6681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:20.347950 6681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:20.348132 6681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:20.355433 6681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:20.355567 6681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:20.355710 6681 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 00:59:20.355798 6681 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:59:20.355908 6681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 00:59:20.365345 6681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:41735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.82.190:32915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:20.366542 6681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:20.367897 6681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:20.381071 6687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:20.381040 6688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:20.381107 6690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:20.381834 6681 server_base.cc:1034] running on GCE node
I20260809 00:59:20.382380 6681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:20.383314 6681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:20.384527 6681 hybrid_clock.cc:648] HybridClock initialized: now 1786237160384489 us; error 48 us; skew 500 ppm
I20260809 00:59:20.384842 6681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:20 dist-test-slave-07c2 krb5kdc[6247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237160, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:20.393388 6681 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 00:59:20.394773 6681 webserver.cc:460] Webserver started at http://127.4.82.131:42185/ using document root <none> and password file <none>
I20260809 00:59:20.395505 6681 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:20.395610 6681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:20.395890 6681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:20.398846 6681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "5092ee8adb0d40698fda6d07e832e5dd"
format_stamp: "Formatted at 2026-08-09 00:59:20 on dist-test-slave-07c2"
server_key: "670df523db2c6effffa074892ff44df8"
server_key_iv: "05209a487eb253cfcd5b3bc02508fb76"
server_key_version: "encryptionkey@0"
I20260809 00:59:20.399623 6681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "5092ee8adb0d40698fda6d07e832e5dd"
format_stamp: "Formatted at 2026-08-09 00:59:20 on dist-test-slave-07c2"
server_key: "670df523db2c6effffa074892ff44df8"
server_key_iv: "05209a487eb253cfcd5b3bc02508fb76"
server_key_version: "encryptionkey@0"
I20260809 00:59:20.404593 6681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 00:59:20.407817 6698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:20.409045 6681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 00:59:20.409206 6681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "5092ee8adb0d40698fda6d07e832e5dd"
format_stamp: "Formatted at 2026-08-09 00:59:20 on dist-test-slave-07c2"
server_key: "670df523db2c6effffa074892ff44df8"
server_key_iv: "05209a487eb253cfcd5b3bc02508fb76"
server_key_version: "encryptionkey@0"
I20260809 00:59:20.409387 6681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:20.423048 6681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:20.426877 6681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:20.427147 6681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:20.429791 6681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:20.429915 6681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:20.430028 6681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:20.430121 6681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:59:20 dist-test-slave-07c2 krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237160, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:20.452816 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.430416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:53229 (local address 127.4.82.190:32915)
0809 00:59:20.430624 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:20.430632 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:20.431986 (+ 1354us) server_negotiation.cc:407] Connection header received
0809 00:59:20.434062 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:20.434069 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:20.434169 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:20.434279 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:20.436448 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.437563 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.440175 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.440884 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.441038 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.444940 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:20.444950 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:20.444958 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:20.445012 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:20.448942 (+ 3930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:20.449938 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:20.449943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:20.449947 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:20.450114 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:20.450708 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:20.450714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:20.450718 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:20.451147 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:20.451340 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:20.452217 (+ 877us) server_negotiation.cc:299] Negotiation successful
0809 00:59:20.452441 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 00:59:20.453863 6708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.430980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:32915 (local address 127.4.82.131:53229)
0809 00:59:20.431832 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:20.431872 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:20.433737 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:20.434518 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:20.434547 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:20.435401 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:20.436256 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.436285 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.437725 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.437731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.439999 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.440017 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.441124 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.441129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.441291 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.441939 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:20.441960 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:20.444671 (+ 2711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:20.449080 (+ 4409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:20.449089 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:20.449110 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:20.449780 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:20.450279 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:20.450284 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:20.450288 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:20.450563 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:20.451328 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:20.451340 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:20.451730 (+ 390us) client_negotiation.cc:769] Sending connection context
0809 00:59:20.452089 (+ 359us) client_negotiation.cc:240] Negotiation successful
0809 00:59:20.452602 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":421,"threads_started":1}
I20260809 00:59:20.469003 6519 heartbeater.cc:503] Master 127.4.82.190:32915 was elected leader, sending a full tablet report...
I20260809 00:59:20.484047 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.466301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:44653 (local address 127.4.82.190:32915)
0809 00:59:20.466498 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:20.466506 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:20.467911 (+ 1405us) server_negotiation.cc:407] Connection header received
0809 00:59:20.468100 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:20.468105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:20.468187 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:20.468285 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:20.469657 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.471159 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.473275 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.474046 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.474224 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.476595 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:20.476604 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:20.476613 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:20.476658 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:20.480403 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:20.481255 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:20.481262 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:20.481269 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:20.481445 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:20.481979 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:20.481984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:20.481988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:20.482380 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:20.482584 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:20.483510 (+ 926us) server_negotiation.cc:299] Negotiation successful
0809 00:59:20.483784 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 00:59:20.484205 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.466639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:32915 (local address 127.4.82.131:44653)
0809 00:59:20.467791 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:20.467826 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:20.467980 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:20.468309 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:20.468314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:20.468863 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:20.469254 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.469267 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.471381 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.471389 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.473101 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.473123 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.474280 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.474288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.474515 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.475194 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:20.475222 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:20.476414 (+ 1192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:20.480557 (+ 4143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:20.480563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:20.480573 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:20.481097 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:20.481609 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:20.481616 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:20.481623 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:20.481820 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:20.482529 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:20.482537 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:20.482843 (+ 306us) client_negotiation.cc:769] Sending connection context
0809 00:59:20.483032 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 00:59:20.483577 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"thread_start_us":824,"threads_started":1}
I20260809 00:59:20.492079 6681 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:38191
I20260809 00:59:20.492125 6838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:38191 every 8 connection(s)
I20260809 00:59:20.493217 6681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 00:59:20.495087 6839 heartbeater.cc:347] Connected to a master server at 127.4.82.190:32915
I20260809 00:59:20.495388 6839 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:20.495994 6839 heartbeater.cc:511] Master 127.4.82.190:32915 requested a full tablet report, sending...
I20260809 00:59:20.496992 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 6681
I20260809 00:59:20.497160 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 00:59:20.497344 6296 ts_manager.cc:194] Registered new tserver with Master: 5092ee8adb0d40698fda6d07e832e5dd (127.4.82.131:38191)
I20260809 00:59:20.497656 4426 external_mini_cluster.cc:1428] Setting key 4d27df09f10644d5d58a5ea305de67d2
I20260809 00:59:20.498250 6296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:53229
I20260809 00:59:20.500610 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 00:59:20 dist-test-slave-07c2 krb5kdc[6247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:59:20 dist-test-slave-07c2 krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:20.549849 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.536734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34208 (local address 127.4.82.190:32915)
0809 00:59:20.536995 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:20.537001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:20.537278 (+ 277us) server_negotiation.cc:407] Connection header received
0809 00:59:20.537432 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:20.537436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:20.537504 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:20.537593 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:20.538937 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.539746 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.540935 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.541343 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.541436 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.545197 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:20.545201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:20.545205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:20.545245 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:20.547645 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:20.548242 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:20.548246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:20.548248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:20.548374 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:20.548702 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:20.548705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:20.548707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:20.548961 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:20.549147 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:20.549407 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 00:59:20.549573 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260809 00:59:20.553964 6295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:59:20.561096 6295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:59:20.587837 6854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.573324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36667 (local address 127.0.0.1:37210)
0809 00:59:20.576983 (+ 3659us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:20.577007 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:20.577165 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:20.578708 (+ 1543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:20.578730 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:20.578775 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:20.579403 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.579421 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.581885 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.581892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.584879 (+ 2987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.584906 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.586847 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.586856 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.587060 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.587114 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 00:59:20.587277 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 00:59:20.587489 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3472,"thread_start_us":3281,"threads_started":1}
I20260809 00:59:20.587886 6856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.573922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37210 (local address 127.4.82.129:36667)
0809 00:59:20.576378 (+ 2456us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:20.576390 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:20.577311 (+ 921us) server_negotiation.cc:407] Connection header received
0809 00:59:20.577444 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:20.577456 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:20.577801 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:20.578038 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:20.580124 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.581713 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.585111 (+ 3398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.586627 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.586783 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.587013 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:20.587401 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 00:59:20.587583 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2236,"thread_start_us":2053,"threads_started":1}
I20260809 00:59:20.589591 6853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.573214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:38191 (local address 127.0.0.1:36814)
0809 00:59:20.574139 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:20.574193 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:20.575574 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:20.580817 (+ 5243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:20.580821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:20.580847 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:20.581251 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.581263 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.584125 (+ 2862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.584131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.586850 (+ 2719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.586865 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.588684 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.588693 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.588902 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.588933 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 00:59:20.589029 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 00:59:20.589178 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":403,"threads_started":1}
I20260809 00:59:20.590528 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.573392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.4.82.131:38191)
0809 00:59:20.580189 (+ 6797us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:20.580199 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:20.580233 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:59:20.580330 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:20.580337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:20.580581 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:20.581437 (+ 856us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:20.581745 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.583950 (+ 2205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.587036 (+ 3086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.588453 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.588624 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.588838 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:20.590031 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0809 00:59:20.590199 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6632,"thread_start_us":6433,"threads_started":1}
I20260809 00:59:20.592015 6444 tablet_service.cc:1467] Processing CreateTablet for tablet 6db60a0a9e77483590c6df8fc4c26e33 (DEFAULT_TABLE table=test-table [id=06b25eb29f07455c88acbf0b6fabac42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:20.593508 6857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.574470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:37683 (local address 127.0.0.1:50306)
0809 00:59:20.585343 (+ 10873us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:20.585363 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:20.585506 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:20.586275 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:20.586280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:20.586303 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:20.586694 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.586725 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.588818 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.588823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.591042 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.591054 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.592754 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.592761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.592948 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.592981 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 00:59:20.593079 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 00:59:20.593191 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10749,"thread_start_us":10591,"threads_started":1}
I20260809 00:59:20.593549 6444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6db60a0a9e77483590c6df8fc4c26e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:20.593757 6858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.574989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50306 (local address 127.4.82.130:37683)
0809 00:59:20.585417 (+ 10428us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:20.585425 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:20.585454 (+ 29us) server_negotiation.cc:407] Connection header received
0809 00:59:20.585913 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:20.585919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:20.586084 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:20.586276 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:20.586947 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.588639 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.591195 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.592516 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.592744 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.592966 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:20.593149 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 00:59:20.593307 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9715,"thread_start_us":9591,"threads_started":1}
I20260809 00:59:20.594058 6756 tablet_service.cc:1467] Processing CreateTablet for tablet 6db60a0a9e77483590c6df8fc4c26e33 (DEFAULT_TABLE table=test-table [id=06b25eb29f07455c88acbf0b6fabac42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:20.595402 6756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6db60a0a9e77483590c6df8fc4c26e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:20.596920 6598 tablet_service.cc:1467] Processing CreateTablet for tablet 6db60a0a9e77483590c6df8fc4c26e33 (DEFAULT_TABLE table=test-table [id=06b25eb29f07455c88acbf0b6fabac42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:20.598205 6598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6db60a0a9e77483590c6df8fc4c26e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:20.608966 6859 tablet_bootstrap.cc:492] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd: Bootstrap starting.
I20260809 00:59:20.610796 6860 tablet_bootstrap.cc:492] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21: Bootstrap starting.
I20260809 00:59:20.613008 6859 tablet_bootstrap.cc:654] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:20.613258 6861 tablet_bootstrap.cc:492] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84: Bootstrap starting.
I20260809 00:59:20.614584 6860 tablet_bootstrap.cc:654] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:20.614637 6859 log.cc:824] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:20.615882 6860 log.cc:824] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:20.616616 6861 tablet_bootstrap.cc:654] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:20.617798 6861 log.cc:824] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:20.618096 6859 tablet_bootstrap.cc:492] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd: No bootstrap required, opened a new log
I20260809 00:59:20.618433 6859 ts_tablet_manager.cc:1397] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260809 00:59:20.619230 6860 tablet_bootstrap.cc:492] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21: No bootstrap required, opened a new log
I20260809 00:59:20.619510 6860 ts_tablet_manager.cc:1397] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 00:59:20.625674 6861 tablet_bootstrap.cc:492] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84: No bootstrap required, opened a new log
I20260809 00:59:20.625918 6861 ts_tablet_manager.cc:1397] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260809 00:59:20.626754 6860 raft_consensus.cc:348] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } }
I20260809 00:59:20.627174 6860 raft_consensus.cc:374] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:20.627302 6860 raft_consensus.cc:729] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa0479418f046048b3cff9079fa4a21, State: Initialized, Role: FOLLOWER
I20260809 00:59:20.627907 6860 consensus_queue.cc:260] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } }
I20260809 00:59:20.629539 6860 ts_tablet_manager.cc:1428] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21: Time spent starting tablet: real 0.010s user 0.002s sys 0.007s
I20260809 00:59:20.630270 6859 raft_consensus.cc:348] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } }
I20260809 00:59:20.630712 6859 raft_consensus.cc:374] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:20.630810 6859 raft_consensus.cc:729] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5092ee8adb0d40698fda6d07e832e5dd, State: Initialized, Role: FOLLOWER
I20260809 00:59:20.631402 6859 consensus_queue.cc:260] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } }
I20260809 00:59:20.632515 6839 heartbeater.cc:503] Master 127.4.82.190:32915 was elected leader, sending a full tablet report...
I20260809 00:59:20.633054 6859 ts_tablet_manager.cc:1428] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd: Time spent starting tablet: real 0.014s user 0.013s sys 0.003s
I20260809 00:59:20.636252 6861 raft_consensus.cc:348] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } }
I20260809 00:59:20.636588 6861 raft_consensus.cc:374] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:20.636709 6861 raft_consensus.cc:729] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5d7a319df74a57888a16783b74df84, State: Initialized, Role: FOLLOWER
I20260809 00:59:20.637264 6861 consensus_queue.cc:260] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } }
I20260809 00:59:20.638165 6676 heartbeater.cc:503] Master 127.4.82.190:32915 was elected leader, sending a full tablet report...
I20260809 00:59:20.638768 6861 ts_tablet_manager.cc:1428] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260809 00:59:20.711709 6520 tablet.cc:2369] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:59:20.736084 6677 tablet.cc:2369] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:59:20.744669 6840 tablet.cc:2369] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:59:20.788781 6865 raft_consensus.cc:482] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:59:20.789121 6865 raft_consensus.cc:504] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } }
I20260809 00:59:20.791329 6865 leader_election.cc:288] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5092ee8adb0d40698fda6d07e832e5dd (127.4.82.131:38191), 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683)
I20260809 00:59:20.798283 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.791763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:32895 (local address 127.4.82.131:38191)
0809 00:59:20.791954 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:20.791959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:20.792618 (+ 659us) server_negotiation.cc:407] Connection header received
0809 00:59:20.792985 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:20.792991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:20.793085 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:20.793140 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:20.793843 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.794552 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.796496 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.797295 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.797484 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.797556 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:20.797858 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 00:59:20.797987 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 00:59:20.798357 6858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.792821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:40597 (local address 127.4.82.130:37683)
0809 00:59:20.793034 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:20.793041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:20.793300 (+ 259us) server_negotiation.cc:407] Connection header received
0809 00:59:20.793500 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:20.793505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:20.793570 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:20.793662 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:20.794194 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.794887 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.796670 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.797437 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.797616 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.797692 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:20.797973 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 00:59:20.798103 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 00:59:20.798673 6869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.792571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:37683 (local address 127.4.82.129:40597)
0809 00:59:20.793181 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:20.793199 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:20.793366 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:20.793692 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:20.793696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:20.793723 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:20.794053 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.794061 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.795025 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.795030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.796520 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.796532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.797662 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.797668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.797826 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.797848 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 00:59:20.797946 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 00:59:20.798067 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":316,"threads_started":1}
I20260809 00:59:20.798710 6868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.791668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:38191 (local address 127.4.82.129:32895)
0809 00:59:20.792498 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:20.792517 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:20.792771 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:20.793319 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:20.793322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:20.793389 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:20.793699 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.793708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.794677 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.794684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.796345 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.796357 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.797499 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.797505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.797673 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.797697 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 00:59:20.797823 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 00:59:20.797949 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":3,"spinlock_wait_cycles":18176,"thread_start_us":426,"threads_started":1}
I20260809 00:59:20.799786 6783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db60a0a9e77483590c6df8fc4c26e33" candidate_uuid: "7aa0479418f046048b3cff9079fa4a21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5092ee8adb0d40698fda6d07e832e5dd" is_pre_election: true
I20260809 00:59:20.799852 6624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db60a0a9e77483590c6df8fc4c26e33" candidate_uuid: "7aa0479418f046048b3cff9079fa4a21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5d7a319df74a57888a16783b74df84" is_pre_election: true
I20260809 00:59:20.800307 6624 raft_consensus.cc:2454] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa0479418f046048b3cff9079fa4a21 in term 0.
I20260809 00:59:20.800303 6783 raft_consensus.cc:2454] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa0479418f046048b3cff9079fa4a21 in term 0.
I20260809 00:59:20.800968 6391 leader_election.cc:302] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5092ee8adb0d40698fda6d07e832e5dd, 7aa0479418f046048b3cff9079fa4a21; no voters:
I20260809 00:59:20.801373 6865 raft_consensus.cc:2781] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:59:20.801523 6865 raft_consensus.cc:482] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:59:20.801641 6865 raft_consensus.cc:3037] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:20.804165 6865 raft_consensus.cc:504] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } }
I20260809 00:59:20.804736 6865 leader_election.cc:288] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [CANDIDATE]: Term 1 election: Requested vote from peers 5092ee8adb0d40698fda6d07e832e5dd (127.4.82.131:38191), 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683)
I20260809 00:59:20.805187 6783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db60a0a9e77483590c6df8fc4c26e33" candidate_uuid: "7aa0479418f046048b3cff9079fa4a21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5092ee8adb0d40698fda6d07e832e5dd"
I20260809 00:59:20.805224 6624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db60a0a9e77483590c6df8fc4c26e33" candidate_uuid: "7aa0479418f046048b3cff9079fa4a21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5d7a319df74a57888a16783b74df84"
I20260809 00:59:20.805392 6783 raft_consensus.cc:3037] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:20.805411 6624 raft_consensus.cc:3037] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:20.807651 6624 raft_consensus.cc:2454] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa0479418f046048b3cff9079fa4a21 in term 1.
I20260809 00:59:20.807715 6783 raft_consensus.cc:2454] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa0479418f046048b3cff9079fa4a21 in term 1.
I20260809 00:59:20.808050 6391 leader_election.cc:302] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5d7a319df74a57888a16783b74df84, 7aa0479418f046048b3cff9079fa4a21; no voters:
I20260809 00:59:20.808311 6865 raft_consensus.cc:2781] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:20.808766 6865 raft_consensus.cc:686] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [term 1 LEADER]: Becoming Leader. State: Replica: 7aa0479418f046048b3cff9079fa4a21, State: Running, Role: LEADER
I20260809 00:59:20.809252 6865 consensus_queue.cc:237] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } }
I20260809 00:59:20.814090 6295 catalog_manager.cc:5495] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aa0479418f046048b3cff9079fa4a21 (127.4.82.129). New cstate: current_term: 1 leader_uuid: "7aa0479418f046048b3cff9079fa4a21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } health_report { overall_health: UNKNOWN } } }
Aug 09 00:59:20 dist-test-slave-07c2 krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237158, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:20.919549 6878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.905973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:32915 (local address 127.0.0.1:34214)
0809 00:59:20.906576 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:20.906591 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:20.906719 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:20.907026 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:20.907029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:20.907822 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:20.908085 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.908095 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.909088 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.909092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.910052 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:20.910063 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.910769 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.910773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:20.910903 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.911677 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:20.911694 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:20.914072 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:20.917015 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:20.917024 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:20.917030 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:20.917441 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:20.917759 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:20.917762 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:20.917765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:20.917904 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:20.918443 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:20.918454 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:20.918664 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 00:59:20.918812 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 00:59:20.919103 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":336,"threads_started":1}
I20260809 00:59:20.919571 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:20.905882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.4.82.190:32915)
0809 00:59:20.906095 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:20.906103 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:20.906686 (+ 583us) server_negotiation.cc:407] Connection header received
0809 00:59:20.906821 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:20.906825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:20.906902 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:20.906984 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:20.908240 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.908966 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.910178 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:20.910611 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:20.910701 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:20.914253 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:20.914258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:20.914261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:20.914287 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:20.916894 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:20.917542 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:20.917546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:20.917548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:20.917664 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:20.917999 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:20.918002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:20.918004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:20.918257 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:20.918440 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:20.918921 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 00:59:20.919259 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":78}
I20260809 00:59:20.924636 6295 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:34214:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 00:59:20.925597 6295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:59:20.932024 6444 tablet_service.cc:1467] Processing CreateTablet for tablet 64bc653e15a449098d2a318798f85886 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1e398a5dd2fb45449316e9e4a1c08673]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:59:20.932260 6598 tablet_service.cc:1467] Processing CreateTablet for tablet 64bc653e15a449098d2a318798f85886 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1e398a5dd2fb45449316e9e4a1c08673]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:59:20.933079 6444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64bc653e15a449098d2a318798f85886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:20.933108 6598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64bc653e15a449098d2a318798f85886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:20.932751 6756 tablet_service.cc:1467] Processing CreateTablet for tablet 64bc653e15a449098d2a318798f85886 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1e398a5dd2fb45449316e9e4a1c08673]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:59:20.933534 6756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64bc653e15a449098d2a318798f85886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:20.938694 6861 tablet_bootstrap.cc:492] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84: Bootstrap starting.
I20260809 00:59:20.940299 6861 tablet_bootstrap.cc:654] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:20.940378 6860 tablet_bootstrap.cc:492] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21: Bootstrap starting.
I20260809 00:59:20.940702 6859 tablet_bootstrap.cc:492] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd: Bootstrap starting.
I20260809 00:59:20.941977 6860 tablet_bootstrap.cc:654] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:20.942353 6859 tablet_bootstrap.cc:654] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:20.942776 6861 tablet_bootstrap.cc:492] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84: No bootstrap required, opened a new log
I20260809 00:59:20.942963 6861 ts_tablet_manager.cc:1397] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 00:59:20.943835 6861 raft_consensus.cc:348] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:20.944327 6861 raft_consensus.cc:374] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:20.944437 6861 raft_consensus.cc:729] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5d7a319df74a57888a16783b74df84, State: Initialized, Role: FOLLOWER
I20260809 00:59:20.944571 6860 tablet_bootstrap.cc:492] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21: No bootstrap required, opened a new log
I20260809 00:59:20.944720 6860 ts_tablet_manager.cc:1397] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 00:59:20.944800 6859 tablet_bootstrap.cc:492] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd: No bootstrap required, opened a new log
I20260809 00:59:20.944687 6861 consensus_queue.cc:260] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:20.944941 6859 ts_tablet_manager.cc:1397] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 00:59:20.945369 6861 ts_tablet_manager.cc:1428] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:59:20.945194 6867 tablet_replica.cc:439] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } }
I20260809 00:59:20.945643 6867 tablet_replica.cc:442] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:20.945567 6860 raft_consensus.cc:348] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:20.945798 6860 raft_consensus.cc:374] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:20.945878 6860 raft_consensus.cc:729] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa0479418f046048b3cff9079fa4a21, State: Initialized, Role: FOLLOWER
I20260809 00:59:20.945770 6859 raft_consensus.cc:348] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:20.946024 6859 raft_consensus.cc:374] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:20.946130 6859 raft_consensus.cc:729] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5092ee8adb0d40698fda6d07e832e5dd, State: Initialized, Role: FOLLOWER
I20260809 00:59:20.946100 6860 consensus_queue.cc:260] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:20.946691 6860 ts_tablet_manager.cc:1428] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 00:59:20.946342 6859 consensus_queue.cc:260] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:20.946622 6865 tablet_replica.cc:439] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } }
I20260809 00:59:20.946935 6865 tablet_replica.cc:442] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:20.947861 6866 tablet_replica.cc:439] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } }
I20260809 00:59:20.948238 6866 tablet_replica.cc:442] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:20.948501 6859 ts_tablet_manager.cc:1428] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 00:59:21.150756 6865 raft_consensus.cc:482] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:59:21.150979 6865 raft_consensus.cc:504] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:21.151695 6865 leader_election.cc:288] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683), 5092ee8adb0d40698fda6d07e832e5dd (127.4.82.131:38191)
I20260809 00:59:21.152412 6783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bc653e15a449098d2a318798f85886" candidate_uuid: "7aa0479418f046048b3cff9079fa4a21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5092ee8adb0d40698fda6d07e832e5dd" is_pre_election: true
I20260809 00:59:21.152415 6624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bc653e15a449098d2a318798f85886" candidate_uuid: "7aa0479418f046048b3cff9079fa4a21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5d7a319df74a57888a16783b74df84" is_pre_election: true
I20260809 00:59:21.152691 6624 raft_consensus.cc:2454] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa0479418f046048b3cff9079fa4a21 in term 0.
I20260809 00:59:21.152671 6783 raft_consensus.cc:2454] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa0479418f046048b3cff9079fa4a21 in term 0.
I20260809 00:59:21.153139 6391 leader_election.cc:302] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5d7a319df74a57888a16783b74df84, 7aa0479418f046048b3cff9079fa4a21; no voters:
I20260809 00:59:21.153412 6865 raft_consensus.cc:2781] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:59:21.153517 6865 raft_consensus.cc:482] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:59:21.153631 6865 raft_consensus.cc:3037] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:21.155687 6865 raft_consensus.cc:504] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:21.156224 6865 leader_election.cc:288] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [CANDIDATE]: Term 1 election: Requested vote from peers 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683), 5092ee8adb0d40698fda6d07e832e5dd (127.4.82.131:38191)
I20260809 00:59:21.156615 6624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bc653e15a449098d2a318798f85886" candidate_uuid: "7aa0479418f046048b3cff9079fa4a21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5d7a319df74a57888a16783b74df84"
I20260809 00:59:21.156698 6783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bc653e15a449098d2a318798f85886" candidate_uuid: "7aa0479418f046048b3cff9079fa4a21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5092ee8adb0d40698fda6d07e832e5dd"
I20260809 00:59:21.156821 6624 raft_consensus.cc:3037] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:21.156963 6783 raft_consensus.cc:3037] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:21.158996 6783 raft_consensus.cc:2454] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa0479418f046048b3cff9079fa4a21 in term 1.
I20260809 00:59:21.159109 6624 raft_consensus.cc:2454] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa0479418f046048b3cff9079fa4a21 in term 1.
I20260809 00:59:21.159407 6391 leader_election.cc:302] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5092ee8adb0d40698fda6d07e832e5dd, 7aa0479418f046048b3cff9079fa4a21; no voters:
I20260809 00:59:21.159694 6865 raft_consensus.cc:2781] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:21.159888 6865 raft_consensus.cc:686] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [term 1 LEADER]: Becoming Leader. State: Replica: 7aa0479418f046048b3cff9079fa4a21, State: Running, Role: LEADER
I20260809 00:59:21.160137 6865 consensus_queue.cc:237] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:21.161651 6870 tablet_replica.cc:439] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21: TxnStatusTablet state changed. Reason: New leader 7aa0479418f046048b3cff9079fa4a21. Latest consensus state: current_term: 1 leader_uuid: "7aa0479418f046048b3cff9079fa4a21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } }
I20260809 00:59:21.161947 6870 tablet_replica.cc:442] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:59:21.162618 6883 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 00:59:21.162895 6883 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 00:59:21.163108 6295 catalog_manager.cc:5495] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aa0479418f046048b3cff9079fa4a21 (127.4.82.129). New cstate: current_term: 1 leader_uuid: "7aa0479418f046048b3cff9079fa4a21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } health_report { overall_health: UNKNOWN } } }
I20260809 00:59:21.190603 6878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.183638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36667 (local address 127.0.0.1:37220)
0809 00:59:21.183908 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:21.183935 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:21.184103 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:21.184815 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:21.184820 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:21.184833 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:59:21.185062 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:21.185071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.186653 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.186657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:21.187688 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:21.187700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.188714 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.188719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:21.188859 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.188912 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:59:21.190028 (+ 1116us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:59:21.190047 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 00:59:21.190115 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 00:59:21.190251 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 00:59:21.190691 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.183664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37220 (local address 127.4.82.129:36667)
0809 00:59:21.184372 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:21.184378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:21.184409 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:59:21.184500 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:21.184505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:21.184600 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:21.184764 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:59:21.185227 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.186510 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.187825 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.188525 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.188677 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.189145 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:59:21.189915 (+ 770us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:59:21.190062 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:21.190193 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 00:59:21.190286 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":366,"threads_started":1}
I20260809 00:59:21.197858 6624 raft_consensus.cc:1261] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Refusing update from remote peer 7aa0479418f046048b3cff9079fa4a21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:21.197966 6783 raft_consensus.cc:1261] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Refusing update from remote peer 7aa0479418f046048b3cff9079fa4a21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:21.198946 6865 consensus_queue.cc:1035] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:21.199316 6870 consensus_queue.cc:1035] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:21.212078 6867 tablet_replica.cc:439] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84: TxnStatusTablet state changed. Reason: New leader 7aa0479418f046048b3cff9079fa4a21. Latest consensus state: current_term: 1 leader_uuid: "7aa0479418f046048b3cff9079fa4a21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } }
I20260809 00:59:21.212414 6867 tablet_replica.cc:442] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:21.215502 6887 mvcc.cc:205] Tried to move back new op lower bound from 7316427412257013760 to 7316427412113883136. Current Snapshot: MvccSnapshot[applied={T|T < 7316427412257013760}]
I20260809 00:59:21.216368 6865 tablet_replica.cc:439] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7aa0479418f046048b3cff9079fa4a21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } }
I20260809 00:59:21.216683 6865 tablet_replica.cc:442] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:59:21.218962 6867 tablet_replica.cc:439] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7aa0479418f046048b3cff9079fa4a21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } }
I20260809 00:59:21.219255 6867 tablet_replica.cc:442] T 64bc653e15a449098d2a318798f85886 P 1f5d7a319df74a57888a16783b74df84: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:21.223248 6866 tablet_replica.cc:439] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd: TxnStatusTablet state changed. Reason: New leader 7aa0479418f046048b3cff9079fa4a21. Latest consensus state: current_term: 1 leader_uuid: "7aa0479418f046048b3cff9079fa4a21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } }
I20260809 00:59:21.223547 6866 tablet_replica.cc:442] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:21.225283 6870 tablet_replica.cc:439] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7aa0479418f046048b3cff9079fa4a21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } }
I20260809 00:59:21.225558 6870 tablet_replica.cc:442] T 64bc653e15a449098d2a318798f85886 P 7aa0479418f046048b3cff9079fa4a21: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:59:21.228678 6866 tablet_replica.cc:439] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7aa0479418f046048b3cff9079fa4a21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } } }
I20260809 00:59:21.228931 6866 tablet_replica.cc:442] T 64bc653e15a449098d2a318798f85886 P 5092ee8adb0d40698fda6d07e832e5dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:59:21.238556 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.228899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37236 (local address 127.4.82.129:36667)
0809 00:59:21.229083 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:21.229090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:21.233619 (+ 4529us) server_negotiation.cc:407] Connection header received
0809 00:59:21.233848 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:21.233853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:21.233929 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:21.234038 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:59:21.234536 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.235402 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.236810 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.237252 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.237390 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.237820 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:59:21.237915 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:59:21.238016 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:21.238181 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 00:59:21.238264 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 00:59:21.254833 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.249026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:46631 (local address 127.4.82.129:36667)
0809 00:59:21.249202 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:21.249206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:21.249821 (+ 615us) server_negotiation.cc:407] Connection header received
0809 00:59:21.250018 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:21.250023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:21.250114 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:21.250221 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:59:21.250712 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.251563 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.253009 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.253449 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.253600 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.254047 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:59:21.254138 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:59:21.254248 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:21.254394 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 00:59:21.254460 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:59:21.254968 6899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.249094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36667 (local address 127.4.82.129:46631)
0809 00:59:21.249718 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:21.249734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:21.249900 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:21.250231 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:21.250235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:21.250245 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:59:21.250568 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:21.250576 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.251686 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.251694 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:21.252864 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:21.252876 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.253655 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.253662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:21.253799 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.253845 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:59:21.254258 (+ 413us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:59:21.254274 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 00:59:21.254365 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 00:59:21.254494 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":347,"threads_started":1}
I20260809 00:59:21.264509 6624 raft_consensus.cc:1261] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Refusing update from remote peer 7aa0479418f046048b3cff9079fa4a21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:21.264524 6783 raft_consensus.cc:1261] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Refusing update from remote peer 7aa0479418f046048b3cff9079fa4a21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:21.265074 6870 consensus_queue.cc:1035] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:21.265419 6865 consensus_queue.cc:1035] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:21.279503 6889 mvcc.cc:205] Tried to move back new op lower bound from 7316427412534665216 to 7316427410678063104. Current Snapshot: MvccSnapshot[applied={T|T < 7316427412534665216}]
I20260809 00:59:21.325874 6295 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:21.326146 6295 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:21.329698 6295 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6: Sending DeleteTablet for 3 replicas of tablet 6db60a0a9e77483590c6df8fc4c26e33
I20260809 00:59:21.330765 6756 tablet_service.cc:1514] Processing DeleteTablet for tablet 6db60a0a9e77483590c6df8fc4c26e33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:21 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:36814
I20260809 00:59:21.331004 6598 tablet_service.cc:1514] Processing DeleteTablet for tablet 6db60a0a9e77483590c6df8fc4c26e33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:21 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:50306
I20260809 00:59:21.331712 6904 tablet_replica.cc:330] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd: stopping tablet replica
I20260809 00:59:21.332257 6904 raft_consensus.cc:2229] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:21.332638 6904 raft_consensus.cc:2258] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:21.332816 6295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:59:21.333506 6295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:59:21.335112 6904 ts_tablet_manager.cc:1905] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:21.338896 6444 tablet_service.cc:1514] Processing DeleteTablet for tablet 6db60a0a9e77483590c6df8fc4c26e33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:21 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:37210
I20260809 00:59:21.341288 6597 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7644b127fa47feaa3e97ab8294a49d (DEFAULT_TABLE table=test-table [id=fb0d526c4e304353a45442468649f969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:21.341825 6597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b7644b127fa47feaa3e97ab8294a49d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:21.341807 6443 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7644b127fa47feaa3e97ab8294a49d (DEFAULT_TABLE table=test-table [id=fb0d526c4e304353a45442468649f969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:21.342330 6443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b7644b127fa47feaa3e97ab8294a49d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:21.344014 6904 ts_tablet_manager.cc:1918] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:59:21.344189 6904 log.cc:1196] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/6db60a0a9e77483590c6df8fc4c26e33
I20260809 00:59:21.344732 6904 ts_tablet_manager.cc:1939] T 6db60a0a9e77483590c6df8fc4c26e33 P 5092ee8adb0d40698fda6d07e832e5dd: Deleting consensus metadata
I20260809 00:59:21.346525 6279 catalog_manager.cc:4841] TS 5092ee8adb0d40698fda6d07e832e5dd (127.4.82.131:38191): tablet 6db60a0a9e77483590c6df8fc4c26e33 (table test-table [id=06b25eb29f07455c88acbf0b6fabac42]) successfully deleted
I20260809 00:59:21.347779 6756 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7644b127fa47feaa3e97ab8294a49d (DEFAULT_TABLE table=test-table [id=fb0d526c4e304353a45442468649f969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:21.348168 6756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b7644b127fa47feaa3e97ab8294a49d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:21.348917 6861 tablet_bootstrap.cc:492] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84: Bootstrap starting.
I20260809 00:59:21.351322 6861 tablet_bootstrap.cc:654] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:21.361394 6859 tablet_bootstrap.cc:492] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd: Bootstrap starting.
I20260809 00:59:21.361959 6860 tablet_bootstrap.cc:492] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21: Bootstrap starting.
I20260809 00:59:21.363184 6859 tablet_bootstrap.cc:654] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:21.363245 6861 tablet_bootstrap.cc:492] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84: No bootstrap required, opened a new log
I20260809 00:59:21.363430 6861 ts_tablet_manager.cc:1397] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.008s
I20260809 00:59:21.363752 6860 tablet_bootstrap.cc:654] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:21.364272 6861 raft_consensus.cc:348] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:21.364606 6861 raft_consensus.cc:374] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:21.364711 6861 raft_consensus.cc:729] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5d7a319df74a57888a16783b74df84, State: Initialized, Role: FOLLOWER
I20260809 00:59:21.365047 6861 consensus_queue.cc:260] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:21.366338 6908 tablet_replica.cc:330] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21: stopping tablet replica
I20260809 00:59:21.366559 6905 tablet_replica.cc:330] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84: stopping tablet replica
I20260809 00:59:21.366807 6908 raft_consensus.cc:2229] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:59:21.367048 6905 raft_consensus.cc:2229] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:21.367441 6908 raft_consensus.cc:2258] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:21.367491 6905 raft_consensus.cc:2258] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:21.368891 6861 ts_tablet_manager.cc:1428] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 00:59:21.371981 6905 ts_tablet_manager.cc:1905] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:21.374766 6859 tablet_bootstrap.cc:492] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd: No bootstrap required, opened a new log
I20260809 00:59:21.374941 6859 ts_tablet_manager.cc:1397] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.005s
I20260809 00:59:21.375954 6859 raft_consensus.cc:348] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:21.376150 6908 ts_tablet_manager.cc:1905] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:21.376199 6859 raft_consensus.cc:374] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:21.376283 6859 raft_consensus.cc:729] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5092ee8adb0d40698fda6d07e832e5dd, State: Initialized, Role: FOLLOWER
I20260809 00:59:21.376570 6859 consensus_queue.cc:260] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:21.377162 6859 ts_tablet_manager.cc:1428] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:59:21.382275 6905 ts_tablet_manager.cc:1918] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:59:21.382522 6905 log.cc:1196] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/6db60a0a9e77483590c6df8fc4c26e33
I20260809 00:59:21.383162 6905 ts_tablet_manager.cc:1939] T 6db60a0a9e77483590c6df8fc4c26e33 P 1f5d7a319df74a57888a16783b74df84: Deleting consensus metadata
I20260809 00:59:21.385130 6279 catalog_manager.cc:4841] TS 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683): tablet 6db60a0a9e77483590c6df8fc4c26e33 (table test-table [id=06b25eb29f07455c88acbf0b6fabac42]) successfully deleted
I20260809 00:59:21.386353 6908 ts_tablet_manager.cc:1918] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:59:21.386533 6908 log.cc:1196] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/6db60a0a9e77483590c6df8fc4c26e33
I20260809 00:59:21.387228 6908 ts_tablet_manager.cc:1939] T 6db60a0a9e77483590c6df8fc4c26e33 P 7aa0479418f046048b3cff9079fa4a21: Deleting consensus metadata
I20260809 00:59:21.388825 6866 raft_consensus.cc:482] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:59:21.389035 6282 catalog_manager.cc:4841] TS 7aa0479418f046048b3cff9079fa4a21 (127.4.82.129:36667): tablet 6db60a0a9e77483590c6df8fc4c26e33 (table test-table [id=06b25eb29f07455c88acbf0b6fabac42]) successfully deleted
I20260809 00:59:21.388999 6866 raft_consensus.cc:504] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:21.389322 6860 tablet_bootstrap.cc:492] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21: No bootstrap required, opened a new log
I20260809 00:59:21.389485 6860 ts_tablet_manager.cc:1397] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.006s
I20260809 00:59:21.390331 6860 raft_consensus.cc:348] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:21.390553 6860 raft_consensus.cc:374] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:21.390671 6860 raft_consensus.cc:729] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa0479418f046048b3cff9079fa4a21, State: Initialized, Role: FOLLOWER
I20260809 00:59:21.390659 6866 leader_election.cc:288] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aa0479418f046048b3cff9079fa4a21 (127.4.82.129:36667), 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683)
I20260809 00:59:21.390944 6860 consensus_queue.cc:260] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:21.391585 6860 ts_tablet_manager.cc:1428] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:59:21.400653 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.391206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:54853 (local address 127.4.82.129:36667)
0809 00:59:21.391515 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:21.391521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:21.392431 (+ 910us) server_negotiation.cc:407] Connection header received
0809 00:59:21.392891 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:21.392896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:21.392969 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:21.393081 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:21.394918 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.396092 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.398260 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.399685 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.399841 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.399916 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:21.400246 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 00:59:21.400391 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260809 00:59:21.401126 6913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.392260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:59139 (local address 127.4.82.130:37683)
0809 00:59:21.393195 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:21.393201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:21.393222 (+ 21us) server_negotiation.cc:407] Connection header received
0809 00:59:21.393294 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:21.393299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:21.393372 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:21.393479 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:21.394039 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.395253 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.398801 (+ 3548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.400083 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.400267 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.400337 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:21.400615 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 00:59:21.400766 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":816,"thread_start_us":1964,"threads_started":1}
I20260809 00:59:21.401433 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.391173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36667 (local address 127.4.82.131:54853)
0809 00:59:21.392326 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:21.392347 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:21.392488 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:21.394367 (+ 1879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:21.394372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:21.394397 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:21.394747 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:21.394760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.396285 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.396290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:21.398021 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:21.398034 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.399862 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.399868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:21.400048 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.400084 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 00:59:21.400893 (+ 809us) client_negotiation.cc:240] Negotiation successful
0809 00:59:21.401030 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"mutex_wait_us":157,"thread_start_us":1552,"threads_started":1}
I20260809 00:59:21.401523 6911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.391698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:37683 (local address 127.4.82.131:59139)
0809 00:59:21.392999 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:21.393017 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:21.393112 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:21.393523 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:21.393528 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:21.393554 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:21.393835 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:21.393847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.395502 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.395507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:21.398607 (+ 3100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:21.398623 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.400295 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.400301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:21.400460 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.400481 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 00:59:21.400588 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 00:59:21.400889 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1167,"mutex_wait_us":25,"thread_start_us":342,"threads_started":1}
I20260809 00:59:21.402192 6624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7644b127fa47feaa3e97ab8294a49d" candidate_uuid: "5092ee8adb0d40698fda6d07e832e5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5d7a319df74a57888a16783b74df84" is_pre_election: true
I20260809 00:59:21.402297 6466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7644b127fa47feaa3e97ab8294a49d" candidate_uuid: "5092ee8adb0d40698fda6d07e832e5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa0479418f046048b3cff9079fa4a21" is_pre_election: true
I20260809 00:59:21.402462 6624 raft_consensus.cc:2454] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5092ee8adb0d40698fda6d07e832e5dd in term 0.
I20260809 00:59:21.402711 6466 raft_consensus.cc:2454] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5092ee8adb0d40698fda6d07e832e5dd in term 0.
I20260809 00:59:21.402981 6699 leader_election.cc:302] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5d7a319df74a57888a16783b74df84, 5092ee8adb0d40698fda6d07e832e5dd; no voters:
I20260809 00:59:21.403417 6866 raft_consensus.cc:2781] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:59:21.403545 6866 raft_consensus.cc:482] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:59:21.403645 6866 raft_consensus.cc:3037] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:21.406138 6866 raft_consensus.cc:504] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:21.406804 6866 leader_election.cc:288] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [CANDIDATE]: Term 1 election: Requested vote from peers 7aa0479418f046048b3cff9079fa4a21 (127.4.82.129:36667), 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683)
I20260809 00:59:21.407433 6466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7644b127fa47feaa3e97ab8294a49d" candidate_uuid: "5092ee8adb0d40698fda6d07e832e5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa0479418f046048b3cff9079fa4a21"
I20260809 00:59:21.407626 6466 raft_consensus.cc:3037] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:21.409868 6466 raft_consensus.cc:2454] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5092ee8adb0d40698fda6d07e832e5dd in term 1.
I20260809 00:59:21.410280 6702 leader_election.cc:302] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5092ee8adb0d40698fda6d07e832e5dd, 7aa0479418f046048b3cff9079fa4a21; no voters:
I20260809 00:59:21.410269 6624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7644b127fa47feaa3e97ab8294a49d" candidate_uuid: "5092ee8adb0d40698fda6d07e832e5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5d7a319df74a57888a16783b74df84"
I20260809 00:59:21.410509 6624 raft_consensus.cc:3037] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:21.410574 6866 raft_consensus.cc:2781] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:21.411072 6866 raft_consensus.cc:686] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [term 1 LEADER]: Becoming Leader. State: Replica: 5092ee8adb0d40698fda6d07e832e5dd, State: Running, Role: LEADER
I20260809 00:59:21.411537 6866 consensus_queue.cc:237] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:21.413156 6624 raft_consensus.cc:2454] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5092ee8adb0d40698fda6d07e832e5dd in term 1.
I20260809 00:59:21.415575 6295 catalog_manager.cc:5495] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5092ee8adb0d40698fda6d07e832e5dd (127.4.82.131). New cstate: current_term: 1 leader_uuid: "5092ee8adb0d40698fda6d07e832e5dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } health_report { overall_health: HEALTHY } } }
I20260809 00:59:21.441506 6917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.433079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36824 (local address 127.4.82.131:38191)
0809 00:59:21.435736 (+ 2657us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:21.435740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:21.435758 (+ 18us) server_negotiation.cc:407] Connection header received
0809 00:59:21.435826 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:21.435829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:21.435892 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:21.436021 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:59:21.436440 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.437203 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.439173 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.439628 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.439730 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.440253 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:59:21.440812 (+ 559us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:59:21.440915 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:21.441082 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 00:59:21.441144 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2559,"thread_start_us":2436,"threads_started":1}
I20260809 00:59:21.456449 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.450887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:50257 (local address 127.4.82.129:36667)
0809 00:59:21.451035 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:21.451039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:21.451635 (+ 596us) server_negotiation.cc:407] Connection header received
0809 00:59:21.451834 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:21.451844 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:21.451907 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:21.451998 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:59:21.452501 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.453245 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.454558 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.455029 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.455191 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.455715 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:59:21.455821 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:59:21.455930 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:21.456036 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 00:59:21.456129 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:59:21.456508 6920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.450877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36667 (local address 127.4.82.131:50257)
0809 00:59:21.451534 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:21.451549 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:21.451706 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:21.452028 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:21.452032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:21.452044 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:59:21.452352 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:21.452360 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.453391 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.453399 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:21.454418 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:21.454429 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.455254 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.455261 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:21.455455 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.455530 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:59:21.455931 (+ 401us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:59:21.455944 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 00:59:21.456019 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 00:59:21.456147 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":310,"threads_started":1}
I20260809 00:59:21.465490 6621 raft_consensus.cc:1261] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Refusing update from remote peer 5092ee8adb0d40698fda6d07e832e5dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:21.466179 6866 consensus_queue.cc:1035] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:21.467571 6466 raft_consensus.cc:1261] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Refusing update from remote peer 5092ee8adb0d40698fda6d07e832e5dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:21.468533 6915 consensus_queue.cc:1035] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:21.569171 6917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.555854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:49161 (local address 127.4.82.131:38191)
0809 00:59:21.556157 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:21.556165 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:21.556194 (+ 29us) server_negotiation.cc:407] Connection header received
0809 00:59:21.556300 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:21.556305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:21.556443 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:21.556587 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:59:21.557702 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.559676 (+ 1974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.566726 (+ 7050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.567526 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.567688 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.568298 (+ 610us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:59:21.568441 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:59:21.568533 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:21.568800 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 00:59:21.568883 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260809 00:59:21.569226 6899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:21.554455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:38191 (local address 127.4.82.129:49161)
0809 00:59:21.555234 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:21.555268 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:21.555491 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:21.556870 (+ 1379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:21.556885 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:21.556908 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:59:21.557255 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:21.557276 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.559983 (+ 2707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.559991 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:21.562949 (+ 2958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:21.562976 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:21.567814 (+ 4838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:21.567820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:21.568038 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:21.568092 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:59:21.568612 (+ 520us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:59:21.568636 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 00:59:21.568746 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 00:59:21.568900 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444}
W20260809 00:59:21.570673 6889 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 00:59:23.170488 6295 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:23.170747 6295 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:23.174594 6295 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6: Sending DeleteTablet for 3 replicas of tablet 5b7644b127fa47feaa3e97ab8294a49d
I20260809 00:59:23.175312 6444 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7644b127fa47feaa3e97ab8294a49d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:23 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:37210
I20260809 00:59:23.175415 6598 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7644b127fa47feaa3e97ab8294a49d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:23 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:50306
I20260809 00:59:23.175506 6756 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7644b127fa47feaa3e97ab8294a49d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:23 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:36814
I20260809 00:59:23.185951 6953 tablet_replica.cc:330] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21: stopping tablet replica
I20260809 00:59:23.186136 6954 tablet_replica.cc:330] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84: stopping tablet replica
I20260809 00:59:23.186309 6953 raft_consensus.cc:2229] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:23.186388 6954 raft_consensus.cc:2229] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:23.186627 6953 raft_consensus.cc:2258] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:23.186676 6954 raft_consensus.cc:2258] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:23.187269 6955 tablet_replica.cc:330] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd: stopping tablet replica
I20260809 00:59:23.187515 6955 raft_consensus.cc:2229] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:59:23.188092 6954 ts_tablet_manager.cc:1905] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:23.188115 6955 raft_consensus.cc:2258] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:23.188139 6953 ts_tablet_manager.cc:1905] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:23.189170 6955 ts_tablet_manager.cc:1905] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:23.195353 6954 ts_tablet_manager.cc:1918] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:59:23.195473 6953 ts_tablet_manager.cc:1918] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:59:23.195555 6954 log.cc:1196] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/5b7644b127fa47feaa3e97ab8294a49d
I20260809 00:59:23.195634 6955 ts_tablet_manager.cc:1918] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:59:23.195663 6953 log.cc:1196] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/5b7644b127fa47feaa3e97ab8294a49d
I20260809 00:59:23.195839 6955 log.cc:1196] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/5b7644b127fa47feaa3e97ab8294a49d
I20260809 00:59:23.196135 6953 ts_tablet_manager.cc:1939] T 5b7644b127fa47feaa3e97ab8294a49d P 7aa0479418f046048b3cff9079fa4a21: Deleting consensus metadata
I20260809 00:59:23.196158 6954 ts_tablet_manager.cc:1939] T 5b7644b127fa47feaa3e97ab8294a49d P 1f5d7a319df74a57888a16783b74df84: Deleting consensus metadata
I20260809 00:59:23.196411 6955 ts_tablet_manager.cc:1939] T 5b7644b127fa47feaa3e97ab8294a49d P 5092ee8adb0d40698fda6d07e832e5dd: Deleting consensus metadata
I20260809 00:59:23.197322 6279 catalog_manager.cc:4841] TS 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683): tablet 5b7644b127fa47feaa3e97ab8294a49d (table test-table [id=fb0d526c4e304353a45442468649f969]) successfully deleted
I20260809 00:59:23.197317 6282 catalog_manager.cc:4841] TS 7aa0479418f046048b3cff9079fa4a21 (127.4.82.129:36667): tablet 5b7644b127fa47feaa3e97ab8294a49d (table test-table [id=fb0d526c4e304353a45442468649f969]) successfully deleted
I20260809 00:59:23.197649 6279 catalog_manager.cc:4841] TS 5092ee8adb0d40698fda6d07e832e5dd (127.4.82.131:38191): tablet 5b7644b127fa47feaa3e97ab8294a49d (table test-table [id=fb0d526c4e304353a45442468649f969]) successfully deleted
Aug 09 00:59:23 dist-test-slave-07c2 krb5kdc[6247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237163, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:59:23 dist-test-slave-07c2 krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237163, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:23.234143 6965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:23.220329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34230 (local address 127.4.82.190:32915)
0809 00:59:23.220898 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:23.220908 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:23.220939 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:59:23.221172 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:23.221177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:23.221245 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:23.221365 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:23.222689 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:23.223636 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:23.224915 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:23.225305 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:23.225446 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:23.229349 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:23.229353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:23.229357 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:23.229388 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:23.232052 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:23.232615 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:23.232619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:23.232621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:23.232758 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:23.233058 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:23.233061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:23.233063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:23.233303 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:23.233478 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:23.233735 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 00:59:23.233890 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":271,"threads_started":1}
I20260809 00:59:23.237497 6295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:59:23.238137 6295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:59:23.244731 6444 tablet_service.cc:1467] Processing CreateTablet for tablet e47ca4ec59b24033acd741a7f4b6703f (DEFAULT_TABLE table=test-table [id=51a93bd8ce5b402f9776d238a998e22d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:23.244805 6598 tablet_service.cc:1467] Processing CreateTablet for tablet e47ca4ec59b24033acd741a7f4b6703f (DEFAULT_TABLE table=test-table [id=51a93bd8ce5b402f9776d238a998e22d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:23.245010 6756 tablet_service.cc:1467] Processing CreateTablet for tablet e47ca4ec59b24033acd741a7f4b6703f (DEFAULT_TABLE table=test-table [id=51a93bd8ce5b402f9776d238a998e22d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:23.245183 6444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e47ca4ec59b24033acd741a7f4b6703f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:23.245230 6598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e47ca4ec59b24033acd741a7f4b6703f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:23.245419 6756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e47ca4ec59b24033acd741a7f4b6703f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:23.251334 6966 tablet_bootstrap.cc:492] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84: Bootstrap starting.
I20260809 00:59:23.252132 6967 tablet_bootstrap.cc:492] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21: Bootstrap starting.
I20260809 00:59:23.252938 6968 tablet_bootstrap.cc:492] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd: Bootstrap starting.
I20260809 00:59:23.253175 6966 tablet_bootstrap.cc:654] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:23.253925 6967 tablet_bootstrap.cc:654] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:23.254666 6968 tablet_bootstrap.cc:654] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:23.256554 6967 tablet_bootstrap.cc:492] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21: No bootstrap required, opened a new log
I20260809 00:59:23.256733 6967 ts_tablet_manager.cc:1397] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 00:59:23.257041 6968 tablet_bootstrap.cc:492] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd: No bootstrap required, opened a new log
I20260809 00:59:23.257176 6968 ts_tablet_manager.cc:1397] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 00:59:23.257470 6967 raft_consensus.cc:348] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.257690 6967 raft_consensus.cc:374] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:23.257814 6967 raft_consensus.cc:729] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa0479418f046048b3cff9079fa4a21, State: Initialized, Role: FOLLOWER
I20260809 00:59:23.257992 6968 raft_consensus.cc:348] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.258229 6968 raft_consensus.cc:374] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:23.258343 6968 raft_consensus.cc:729] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5092ee8adb0d40698fda6d07e832e5dd, State: Initialized, Role: FOLLOWER
I20260809 00:59:23.258055 6967 consensus_queue.cc:260] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.258574 6968 consensus_queue.cc:260] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.259099 6967 ts_tablet_manager.cc:1428] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:59:23.259207 6968 ts_tablet_manager.cc:1428] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:59:23.260166 6966 tablet_bootstrap.cc:492] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84: No bootstrap required, opened a new log
I20260809 00:59:23.260325 6966 ts_tablet_manager.cc:1397] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.004s
I20260809 00:59:23.261174 6966 raft_consensus.cc:348] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.261369 6966 raft_consensus.cc:374] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:23.261489 6966 raft_consensus.cc:729] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5d7a319df74a57888a16783b74df84, State: Initialized, Role: FOLLOWER
I20260809 00:59:23.261718 6966 consensus_queue.cc:260] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.262579 6966 ts_tablet_manager.cc:1428] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 00:59:23.291620 6930 raft_consensus.cc:482] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:59:23.291833 6930 raft_consensus.cc:504] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.292512 6930 leader_election.cc:288] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683), 5092ee8adb0d40698fda6d07e832e5dd (127.4.82.131:38191)
I20260809 00:59:23.293054 6621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47ca4ec59b24033acd741a7f4b6703f" candidate_uuid: "7aa0479418f046048b3cff9079fa4a21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5d7a319df74a57888a16783b74df84" is_pre_election: true
I20260809 00:59:23.293143 6782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47ca4ec59b24033acd741a7f4b6703f" candidate_uuid: "7aa0479418f046048b3cff9079fa4a21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5092ee8adb0d40698fda6d07e832e5dd" is_pre_election: true
I20260809 00:59:23.293330 6621 raft_consensus.cc:2454] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa0479418f046048b3cff9079fa4a21 in term 0.
I20260809 00:59:23.293465 6782 raft_consensus.cc:2454] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aa0479418f046048b3cff9079fa4a21 in term 0.
I20260809 00:59:23.293915 6391 leader_election.cc:302] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5d7a319df74a57888a16783b74df84, 7aa0479418f046048b3cff9079fa4a21; no voters:
I20260809 00:59:23.294209 6930 raft_consensus.cc:2781] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:59:23.294338 6930 raft_consensus.cc:482] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:59:23.294450 6930 raft_consensus.cc:3037] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:23.296759 6930 raft_consensus.cc:504] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.297297 6930 leader_election.cc:288] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [CANDIDATE]: Term 1 election: Requested vote from peers 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683), 5092ee8adb0d40698fda6d07e832e5dd (127.4.82.131:38191)
I20260809 00:59:23.297856 6621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47ca4ec59b24033acd741a7f4b6703f" candidate_uuid: "7aa0479418f046048b3cff9079fa4a21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5d7a319df74a57888a16783b74df84"
I20260809 00:59:23.297881 6782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47ca4ec59b24033acd741a7f4b6703f" candidate_uuid: "7aa0479418f046048b3cff9079fa4a21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5092ee8adb0d40698fda6d07e832e5dd"
I20260809 00:59:23.298076 6782 raft_consensus.cc:3037] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:23.298074 6621 raft_consensus.cc:3037] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:23.300145 6782 raft_consensus.cc:2454] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa0479418f046048b3cff9079fa4a21 in term 1.
I20260809 00:59:23.300374 6621 raft_consensus.cc:2454] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aa0479418f046048b3cff9079fa4a21 in term 1.
I20260809 00:59:23.300575 6391 leader_election.cc:302] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5092ee8adb0d40698fda6d07e832e5dd, 7aa0479418f046048b3cff9079fa4a21; no voters:
I20260809 00:59:23.300894 6930 raft_consensus.cc:2781] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:23.301019 6930 raft_consensus.cc:686] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [term 1 LEADER]: Becoming Leader. State: Replica: 7aa0479418f046048b3cff9079fa4a21, State: Running, Role: LEADER
I20260809 00:59:23.301317 6930 consensus_queue.cc:237] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.304035 6295 catalog_manager.cc:5495] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aa0479418f046048b3cff9079fa4a21 (127.4.82.129). New cstate: current_term: 1 leader_uuid: "7aa0479418f046048b3cff9079fa4a21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } health_report { overall_health: UNKNOWN } } }
I20260809 00:59:23.325644 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:23.319899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37246 (local address 127.4.82.129:36667)
0809 00:59:23.320396 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:23.320400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:23.320418 (+ 18us) server_negotiation.cc:407] Connection header received
0809 00:59:23.320465 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:23.320469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:23.320526 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:23.320610 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:59:23.320886 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:23.321626 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:23.323198 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:23.323968 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:23.324144 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:23.324669 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:59:23.324872 (+ 203us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:59:23.325010 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:23.325150 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 00:59:23.325230 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":287,"threads_started":1}
I20260809 00:59:23.334348 6621 raft_consensus.cc:1261] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Refusing update from remote peer 7aa0479418f046048b3cff9079fa4a21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:23.334331 6782 raft_consensus.cc:1261] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Refusing update from remote peer 7aa0479418f046048b3cff9079fa4a21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:23.334841 6930 consensus_queue.cc:1035] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:23.335181 6947 consensus_queue.cc:1035] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:23.373629 6296 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:23.373878 6296 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:23.376703 6296 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6: Sending DeleteTablet for 3 replicas of tablet e47ca4ec59b24033acd741a7f4b6703f
I20260809 00:59:23.377369 6443 tablet_service.cc:1514] Processing DeleteTablet for tablet e47ca4ec59b24033acd741a7f4b6703f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:23 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:37210
I20260809 00:59:23.377497 6598 tablet_service.cc:1514] Processing DeleteTablet for tablet e47ca4ec59b24033acd741a7f4b6703f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:23 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:50306
I20260809 00:59:23.377604 6756 tablet_service.cc:1514] Processing DeleteTablet for tablet e47ca4ec59b24033acd741a7f4b6703f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:23 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:36814
I20260809 00:59:23.377737 6954 tablet_replica.cc:330] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84: stopping tablet replica
I20260809 00:59:23.377902 6953 tablet_replica.cc:330] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21: stopping tablet replica
I20260809 00:59:23.377904 6955 tablet_replica.cc:330] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd: stopping tablet replica
I20260809 00:59:23.377947 6954 raft_consensus.cc:2229] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:23.378094 6955 raft_consensus.cc:2229] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:23.378131 6954 raft_consensus.cc:2258] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:23.378091 6953 raft_consensus.cc:2229] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:59:23.378285 6955 raft_consensus.cc:2258] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:23.378425 6953 raft_consensus.cc:2258] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:23.378618 6296 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 00:59:23.379182 6955 ts_tablet_manager.cc:1905] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 00:59:23.379419 6296 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:59:23.379587 6954 ts_tablet_manager.cc:1905] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:23.386020 6955 ts_tablet_manager.cc:1918] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:59:23.386020 6954 ts_tablet_manager.cc:1918] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:59:23.386178 6954 log.cc:1196] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/e47ca4ec59b24033acd741a7f4b6703f
I20260809 00:59:23.386178 6955 log.cc:1196] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/e47ca4ec59b24033acd741a7f4b6703f
I20260809 00:59:23.386621 6955 ts_tablet_manager.cc:1939] T e47ca4ec59b24033acd741a7f4b6703f P 5092ee8adb0d40698fda6d07e832e5dd: Deleting consensus metadata
I20260809 00:59:23.386683 6954 ts_tablet_manager.cc:1939] T e47ca4ec59b24033acd741a7f4b6703f P 1f5d7a319df74a57888a16783b74df84: Deleting consensus metadata
I20260809 00:59:23.386833 6953 ts_tablet_manager.cc:1905] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:23.387791 6279 catalog_manager.cc:4841] TS 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683): tablet e47ca4ec59b24033acd741a7f4b6703f (table test-table [id=51a93bd8ce5b402f9776d238a998e22d]) successfully deleted
I20260809 00:59:23.388540 6279 catalog_manager.cc:4841] TS 5092ee8adb0d40698fda6d07e832e5dd (127.4.82.131:38191): tablet e47ca4ec59b24033acd741a7f4b6703f (table test-table [id=51a93bd8ce5b402f9776d238a998e22d]) successfully deleted
I20260809 00:59:23.391168 6598 tablet_service.cc:1467] Processing CreateTablet for tablet 9feb07d64c23497e8697db9f30b5d543 (DEFAULT_TABLE table=test-table [id=591d5e4e237340129f03773040ce8f37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:23.391408 6443 tablet_service.cc:1467] Processing CreateTablet for tablet 9feb07d64c23497e8697db9f30b5d543 (DEFAULT_TABLE table=test-table [id=591d5e4e237340129f03773040ce8f37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:23.391600 6598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9feb07d64c23497e8697db9f30b5d543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:23.391775 6443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9feb07d64c23497e8697db9f30b5d543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:23.393569 6953 ts_tablet_manager.cc:1918] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:59:23.393728 6953 log.cc:1196] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/e47ca4ec59b24033acd741a7f4b6703f
I20260809 00:59:23.394173 6953 ts_tablet_manager.cc:1939] T e47ca4ec59b24033acd741a7f4b6703f P 7aa0479418f046048b3cff9079fa4a21: Deleting consensus metadata
I20260809 00:59:23.395191 6282 catalog_manager.cc:4841] TS 7aa0479418f046048b3cff9079fa4a21 (127.4.82.129:36667): tablet e47ca4ec59b24033acd741a7f4b6703f (table test-table [id=51a93bd8ce5b402f9776d238a998e22d]) successfully deleted
I20260809 00:59:23.395378 6756 tablet_service.cc:1467] Processing CreateTablet for tablet 9feb07d64c23497e8697db9f30b5d543 (DEFAULT_TABLE table=test-table [id=591d5e4e237340129f03773040ce8f37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:23.395798 6756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9feb07d64c23497e8697db9f30b5d543. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:23.397702 6966 tablet_bootstrap.cc:492] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84: Bootstrap starting.
I20260809 00:59:23.398962 6966 tablet_bootstrap.cc:654] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:23.399479 6967 tablet_bootstrap.cc:492] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21: Bootstrap starting.
I20260809 00:59:23.400974 6967 tablet_bootstrap.cc:654] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:23.400998 6966 tablet_bootstrap.cc:492] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84: No bootstrap required, opened a new log
I20260809 00:59:23.401110 6966 ts_tablet_manager.cc:1397] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 00:59:23.401623 6966 raft_consensus.cc:348] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.401893 6966 raft_consensus.cc:374] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:23.401962 6966 raft_consensus.cc:729] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5d7a319df74a57888a16783b74df84, State: Initialized, Role: FOLLOWER
I20260809 00:59:23.402189 6966 consensus_queue.cc:260] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.402861 6966 ts_tablet_manager.cc:1428] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 00:59:23.404008 6968 tablet_bootstrap.cc:492] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd: Bootstrap starting.
I20260809 00:59:23.404978 6967 tablet_bootstrap.cc:492] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21: No bootstrap required, opened a new log
I20260809 00:59:23.405119 6967 ts_tablet_manager.cc:1397] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260809 00:59:23.405658 6968 tablet_bootstrap.cc:654] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:23.405896 6967 raft_consensus.cc:348] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.406160 6967 raft_consensus.cc:374] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:23.406248 6967 raft_consensus.cc:729] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa0479418f046048b3cff9079fa4a21, State: Initialized, Role: FOLLOWER
I20260809 00:59:23.406484 6967 consensus_queue.cc:260] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.407029 6967 ts_tablet_manager.cc:1428] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 00:59:23.408120 6968 tablet_bootstrap.cc:492] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd: No bootstrap required, opened a new log
I20260809 00:59:23.408301 6968 ts_tablet_manager.cc:1397] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 00:59:23.408977 6968 raft_consensus.cc:348] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.409232 6968 raft_consensus.cc:374] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:23.409328 6968 raft_consensus.cc:729] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5092ee8adb0d40698fda6d07e832e5dd, State: Initialized, Role: FOLLOWER
I20260809 00:59:23.409597 6968 consensus_queue.cc:260] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.410157 6968 ts_tablet_manager.cc:1428] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:59:23.556027 6944 raft_consensus.cc:482] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:59:23.556284 6944 raft_consensus.cc:504] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.556972 6944 leader_election.cc:288] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aa0479418f046048b3cff9079fa4a21 (127.4.82.129:36667), 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683)
I20260809 00:59:23.557622 6621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9feb07d64c23497e8697db9f30b5d543" candidate_uuid: "5092ee8adb0d40698fda6d07e832e5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5d7a319df74a57888a16783b74df84" is_pre_election: true
I20260809 00:59:23.557643 6466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9feb07d64c23497e8697db9f30b5d543" candidate_uuid: "5092ee8adb0d40698fda6d07e832e5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa0479418f046048b3cff9079fa4a21" is_pre_election: true
I20260809 00:59:23.557938 6466 raft_consensus.cc:2454] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5092ee8adb0d40698fda6d07e832e5dd in term 0.
I20260809 00:59:23.557931 6621 raft_consensus.cc:2454] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5092ee8adb0d40698fda6d07e832e5dd in term 0.
I20260809 00:59:23.558404 6699 leader_election.cc:302] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5d7a319df74a57888a16783b74df84, 5092ee8adb0d40698fda6d07e832e5dd; no voters:
I20260809 00:59:23.558728 6944 raft_consensus.cc:2781] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:59:23.558866 6944 raft_consensus.cc:482] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:59:23.558965 6944 raft_consensus.cc:3037] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:23.561111 6944 raft_consensus.cc:504] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.561750 6944 leader_election.cc:288] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [CANDIDATE]: Term 1 election: Requested vote from peers 7aa0479418f046048b3cff9079fa4a21 (127.4.82.129:36667), 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683)
I20260809 00:59:23.562224 6466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9feb07d64c23497e8697db9f30b5d543" candidate_uuid: "5092ee8adb0d40698fda6d07e832e5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aa0479418f046048b3cff9079fa4a21"
I20260809 00:59:23.562289 6621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9feb07d64c23497e8697db9f30b5d543" candidate_uuid: "5092ee8adb0d40698fda6d07e832e5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5d7a319df74a57888a16783b74df84"
I20260809 00:59:23.562427 6466 raft_consensus.cc:3037] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:23.562577 6621 raft_consensus.cc:3037] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:23.564487 6466 raft_consensus.cc:2454] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5092ee8adb0d40698fda6d07e832e5dd in term 1.
I20260809 00:59:23.564692 6621 raft_consensus.cc:2454] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5092ee8adb0d40698fda6d07e832e5dd in term 1.
I20260809 00:59:23.564939 6702 leader_election.cc:302] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5092ee8adb0d40698fda6d07e832e5dd, 7aa0479418f046048b3cff9079fa4a21; no voters:
I20260809 00:59:23.565258 6944 raft_consensus.cc:2781] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:23.565375 6944 raft_consensus.cc:686] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [term 1 LEADER]: Becoming Leader. State: Replica: 5092ee8adb0d40698fda6d07e832e5dd, State: Running, Role: LEADER
I20260809 00:59:23.565675 6944 consensus_queue.cc:237] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } }
I20260809 00:59:23.568465 6296 catalog_manager.cc:5495] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5092ee8adb0d40698fda6d07e832e5dd (127.4.82.131). New cstate: current_term: 1 leader_uuid: "5092ee8adb0d40698fda6d07e832e5dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5092ee8adb0d40698fda6d07e832e5dd" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38191 } health_report { overall_health: HEALTHY } } }
I20260809 00:59:23.601228 6985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:23.596394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36830 (local address 127.4.82.131:38191)
0809 00:59:23.597013 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:23.597018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:23.597034 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:59:23.597096 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:23.597099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:23.597153 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:23.597228 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:59:23.597528 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:23.598273 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:23.599713 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:23.600144 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:23.600250 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:23.600637 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:59:23.600734 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:59:23.600802 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:23.600924 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 00:59:23.600994 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":373,"threads_started":1}
I20260809 00:59:23.609294 6621 raft_consensus.cc:1261] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Refusing update from remote peer 5092ee8adb0d40698fda6d07e832e5dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:23.609313 6466 raft_consensus.cc:1261] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Refusing update from remote peer 5092ee8adb0d40698fda6d07e832e5dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:23.609854 6944 consensus_queue.cc:1035] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f5d7a319df74a57888a16783b74df84" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 37683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:23.610101 6924 consensus_queue.cc:1035] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aa0479418f046048b3cff9079fa4a21" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 00:59:23.630021 6931 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 00:59:25.205572 6296 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:25.205830 6296 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:25.209836 6296 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f38c035462e47f1b87fc53b68e5a4d6: Sending DeleteTablet for 3 replicas of tablet 9feb07d64c23497e8697db9f30b5d543
I20260809 00:59:25.210588 6443 tablet_service.cc:1514] Processing DeleteTablet for tablet 9feb07d64c23497e8697db9f30b5d543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:25 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:37210
I20260809 00:59:25.210642 6598 tablet_service.cc:1514] Processing DeleteTablet for tablet 9feb07d64c23497e8697db9f30b5d543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:25 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:50306
I20260809 00:59:25.210665 6756 tablet_service.cc:1514] Processing DeleteTablet for tablet 9feb07d64c23497e8697db9f30b5d543 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:25 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:36814
I20260809 00:59:25.211345 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 6374
I20260809 00:59:25.223521 7015 tablet_replica.cc:330] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21: stopping tablet replica
I20260809 00:59:25.223744 7016 tablet_replica.cc:330] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd: stopping tablet replica
I20260809 00:59:25.223871 7015 raft_consensus.cc:2229] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:25.223966 7017 tablet_replica.cc:330] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84: stopping tablet replica
I20260809 00:59:25.224066 7016 raft_consensus.cc:2229] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:59:25.224161 7017 raft_consensus.cc:2229] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:25.224282 7015 raft_consensus.cc:2258] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:25.224414 7017 raft_consensus.cc:2258] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:25.224565 7016 raft_consensus.cc:2258] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:25.225572 7017 ts_tablet_manager.cc:1905] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:25.225636 7016 ts_tablet_manager.cc:1905] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:25.226533 7015 ts_tablet_manager.cc:1905] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:25.232445 7016 ts_tablet_manager.cc:1918] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:59:25.232445 7017 ts_tablet_manager.cc:1918] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:59:25.232589 7017 log.cc:1196] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/9feb07d64c23497e8697db9f30b5d543
I20260809 00:59:25.232594 7016 log.cc:1196] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/9feb07d64c23497e8697db9f30b5d543
I20260809 00:59:25.233053 7016 ts_tablet_manager.cc:1939] T 9feb07d64c23497e8697db9f30b5d543 P 5092ee8adb0d40698fda6d07e832e5dd: Deleting consensus metadata
I20260809 00:59:25.233084 7017 ts_tablet_manager.cc:1939] T 9feb07d64c23497e8697db9f30b5d543 P 1f5d7a319df74a57888a16783b74df84: Deleting consensus metadata
I20260809 00:59:25.234233 6279 catalog_manager.cc:4841] TS 1f5d7a319df74a57888a16783b74df84 (127.4.82.130:37683): tablet 9feb07d64c23497e8697db9f30b5d543 (table test-table [id=591d5e4e237340129f03773040ce8f37]) successfully deleted
I20260809 00:59:25.234477 6279 catalog_manager.cc:4841] TS 5092ee8adb0d40698fda6d07e832e5dd (127.4.82.131:38191): tablet 9feb07d64c23497e8697db9f30b5d543 (table test-table [id=591d5e4e237340129f03773040ce8f37]) successfully deleted
I20260809 00:59:25.234973 7015 ts_tablet_manager.cc:1918] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:59:25.235134 7015 log.cc:1196] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/9feb07d64c23497e8697db9f30b5d543
I20260809 00:59:25.235783 7015 ts_tablet_manager.cc:1939] T 9feb07d64c23497e8697db9f30b5d543 P 7aa0479418f046048b3cff9079fa4a21: Deleting consensus metadata
I20260809 00:59:25.236869 6282 catalog_manager.cc:4841] TS 7aa0479418f046048b3cff9079fa4a21 (127.4.82.129:36667): tablet 9feb07d64c23497e8697db9f30b5d543 (table test-table [id=591d5e4e237340129f03773040ce8f37]) successfully deleted
Aug 09 00:59:25 dist-test-slave-07c2 krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237163, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 00:59:25.247225 7019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:25.214978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37262 (local address 127.4.82.129:36667)
0809 00:59:25.225728 (+ 10750us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:25.225735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:25.231852 (+ 6117us) server_negotiation.cc:407] Connection header received
0809 00:59:25.232062 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:25.232070 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:25.232158 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:25.232264 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:25.235336 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:25.236584 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:25.237887 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:25.238295 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:25.238390 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:25.242253 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:25.242267 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:25.242283 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:25.242318 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:25.245019 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:25.245585 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:25.245591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:25.245593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:25.245744 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:25.246039 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:25.246042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:25.246044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:25.246360 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:25.246581 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:25.246814 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 00:59:25.246980 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10607,"thread_start_us":10676,"threads_started":1}
W20260809 00:59:25.247952 6513 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37262
W20260809 00:59:25.248507 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 6374: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 00:59:25.248761 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 6374
I20260809 00:59:25.277690 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 6524
Aug 09 00:59:25 dist-test-slave-07c2 krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237163, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 00:59:25.292452 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:25.278505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50322 (local address 127.4.82.130:37683)
0809 00:59:25.279027 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:25.279031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:25.279047 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:59:25.279100 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:25.279103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:25.279161 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:25.279250 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:25.280514 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:25.281315 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:25.282409 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:25.282780 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:25.282870 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:25.286957 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:25.286970 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:25.286984 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:25.287015 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:25.289878 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:25.290592 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:25.290599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:25.290601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:25.290751 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:25.291044 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:25.291047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:25.291048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:25.291436 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:25.291653 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:25.291908 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 00:59:25.292073 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":279,"threads_started":1}
W20260809 00:59:25.293164 6669 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50322
W20260809 00:59:25.293751 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 6524: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 00:59:25.293921 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 6524
I20260809 00:59:25.314146 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 6681
Aug 09 00:59:25 dist-test-slave-07c2 krb5kdc[6247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237163, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 00:59:25.328739 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:25.314907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36840 (local address 127.4.82.131:38191)
0809 00:59:25.315413 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:25.315417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:25.315434 (+ 17us) server_negotiation.cc:407] Connection header received
0809 00:59:25.315485 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:25.315489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:25.315552 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:25.315646 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:25.316836 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:25.317596 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:25.318684 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:25.319060 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:25.319146 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:25.323555 (+ 4409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:25.323569 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:25.323585 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:25.323613 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:25.326247 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:25.326824 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:25.326830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:25.326832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:25.326970 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:25.327334 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:25.327341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:25.327346 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:25.327745 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:25.327951 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:25.328178 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 00:59:25.328348 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":236,"threads_started":1}
W20260809 00:59:25.329478 6831 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36840
W20260809 00:59:25.330152 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 6681: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 00:59:25.330344 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 6681
I20260809 00:59:25.355341 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 6262
I20260809 00:59:25.356453 6341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:25.471844 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 6262
2026-08-09T00:59:25Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8402 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:59:25 dist-test-slave-07c2 krb5kdc[7026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:59:25 dist-test-slave-07c2 krb5kdc[7026](info): set up 2 sockets
Aug 09 00:59:25 dist-test-slave-07c2 krb5kdc[7026](info): commencing operation
krb5kdc: starting...
W20260809 00:59:26.586848 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 00:59:26 dist-test-slave-07c2 krb5kdc[7026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:59:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 00:59:26.779162 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:35513
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:35513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:27.092986 7039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:27.093245 7039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:27.093426 7039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:27.100271 7039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:27.100363 7039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:27.100415 7039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:27.100471 7039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:27.100513 7039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:27.109841 7039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:35513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:35513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:27.111013 7039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:27.112267 7039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:27.123042 7044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:27.123416 7045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:27.123962 7047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:27.125120 7039 server_base.cc:1034] running on GCE node
I20260809 00:59:27.125815 7039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:27.127048 7039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:27.128278 7039 hybrid_clock.cc:648] HybridClock initialized: now 1786237167128229 us; error 39 us; skew 500 ppm
I20260809 00:59:27.128580 7039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:27 dist-test-slave-07c2 krb5kdc[7026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237167, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:27.131661 7039 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 00:59:27.133090 7039 webserver.cc:460] Webserver started at http://127.4.82.190:41391/ using document root <none> and password file <none>
I20260809 00:59:27.133782 7039 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:27.133919 7039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:27.134243 7039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:27.137173 7039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "3ca7f2096c2043dc94d3000eb86b74a5"
format_stamp: "Formatted at 2026-08-09 00:59:27 on dist-test-slave-07c2"
server_key: "c63d93235b0f051c9d989bb20ed90a6e"
server_key_iv: "9e5b91f879e15061bd439484d775682b"
server_key_version: "encryptionkey@0"
I20260809 00:59:27.137950 7039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "3ca7f2096c2043dc94d3000eb86b74a5"
format_stamp: "Formatted at 2026-08-09 00:59:27 on dist-test-slave-07c2"
server_key: "c63d93235b0f051c9d989bb20ed90a6e"
server_key_iv: "9e5b91f879e15061bd439484d775682b"
server_key_version: "encryptionkey@0"
I20260809 00:59:27.142952 7039 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 00:59:27.146323 7056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:27.147639 7039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 00:59:27.147827 7039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "3ca7f2096c2043dc94d3000eb86b74a5"
format_stamp: "Formatted at 2026-08-09 00:59:27 on dist-test-slave-07c2"
server_key: "c63d93235b0f051c9d989bb20ed90a6e"
server_key_iv: "9e5b91f879e15061bd439484d775682b"
server_key_version: "encryptionkey@0"
I20260809 00:59:27.148023 7039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:27.181052 7039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:27.185141 7039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:27.185453 7039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:27.203666 7039 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:35513
I20260809 00:59:27.203701 7113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:35513 every 8 connection(s)
I20260809 00:59:27.205063 7039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:27.205691 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 7039
I20260809 00:59:27.205967 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:27.206785 4426 external_mini_cluster.cc:1428] Setting key ec17b90971252f36b7b2b19824f32044
I20260809 00:59:27.209697 7114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 00:59:27 dist-test-slave-07c2 krb5kdc[7026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:27.222872 7114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5: Bootstrap starting.
I20260809 00:59:27.226112 7114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:27.227101 7114 log.cc:824] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:27.229945 7118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:27.208883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51628 (local address 127.4.82.190:35513)
0809 00:59:27.209670 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:27.209685 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:27.209730 (+ 45us) server_negotiation.cc:407] Connection header received
0809 00:59:27.211495 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:27.211516 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:27.211868 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:27.212173 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:27.213963 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:27.215365 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:27.217217 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:27.217948 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:27.218071 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:27.222768 (+ 4697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:27.222776 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:27.222790 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:27.222824 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:27.225940 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:27.226682 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:27.226689 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:27.226695 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:27.226867 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:27.227256 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:27.227264 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:27.227268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:27.227828 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:27.228131 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:27.228623 (+ 492us) server_negotiation.cc:299] Negotiation successful
0809 00:59:27.229017 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":390,"threads_started":1}
I20260809 00:59:27.229976 7114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5: No bootstrap required, opened a new log
I20260809 00:59:27.237869 7114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca7f2096c2043dc94d3000eb86b74a5" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 35513 } }
I20260809 00:59:27.238255 7114 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:27.238374 7114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ca7f2096c2043dc94d3000eb86b74a5, State: Initialized, Role: FOLLOWER
I20260809 00:59:27.238889 7114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca7f2096c2043dc94d3000eb86b74a5" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 35513 } }
I20260809 00:59:27.239133 7114 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:27.239346 7114 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:27.239540 7114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:27.241695 7114 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca7f2096c2043dc94d3000eb86b74a5" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 35513 } }
I20260809 00:59:27.242179 7114 leader_election.cc:302] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ca7f2096c2043dc94d3000eb86b74a5; no voters:
I20260809 00:59:27.242868 7114 leader_election.cc:288] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:27.242996 7121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:27.243938 7121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [term 1 LEADER]: Becoming Leader. State: Replica: 3ca7f2096c2043dc94d3000eb86b74a5, State: Running, Role: LEADER
I20260809 00:59:27.244457 7121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca7f2096c2043dc94d3000eb86b74a5" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 35513 } }
I20260809 00:59:27.245018 7114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:27.247788 7123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ca7f2096c2043dc94d3000eb86b74a5. Latest consensus state: current_term: 1 leader_uuid: "3ca7f2096c2043dc94d3000eb86b74a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca7f2096c2043dc94d3000eb86b74a5" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 35513 } } }
I20260809 00:59:27.247783 7122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ca7f2096c2043dc94d3000eb86b74a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca7f2096c2043dc94d3000eb86b74a5" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 35513 } } }
I20260809 00:59:27.248219 7122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:27.248193 7123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:27.250105 7131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:27.258606 7131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:27.267275 7131 catalog_manager.cc:1347] Generated new cluster ID: a64dc79c4326408e9ffca0600d56d460
I20260809 00:59:27.267421 7131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:27.285184 7131 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:27.286513 7131 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:27.294298 7131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ca7f2096c2043dc94d3000eb86b74a5: Generated new TSK 0
I20260809 00:59:27.295073 7131 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 00:59:27.342790 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:35513
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:27.668016 7143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:27.668289 7143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:27.668394 7143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:27.675222 7143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:27.675374 7143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:27.675482 7143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 00:59:27.687314 7143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:35513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:27.689110 7143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:27.690768 7143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:27.703529 7149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:27.703614 7148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:27.703596 7152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:27.704562 7143 server_base.cc:1034] running on GCE node
I20260809 00:59:27.705076 7143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:27.705955 7143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:27.707154 7143 hybrid_clock.cc:648] HybridClock initialized: now 1786237167707116 us; error 35 us; skew 500 ppm
I20260809 00:59:27.707480 7143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:27 dist-test-slave-07c2 krb5kdc[7026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237167, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:27.710501 7143 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 00:59:27.712029 7143 webserver.cc:460] Webserver started at http://127.4.82.129:35757/ using document root <none> and password file <none>
I20260809 00:59:27.712677 7143 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:27.712811 7143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:27.713124 7143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:27.715958 7143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "e5eba24237404f40a3533c3c393acc87"
format_stamp: "Formatted at 2026-08-09 00:59:27 on dist-test-slave-07c2"
server_key: "3bf865abd9310a5fe2c71b3efe5a48d6"
server_key_iv: "19253028c62fb5c10c068b8c0375422a"
server_key_version: "encryptionkey@0"
I20260809 00:59:27.716732 7143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "e5eba24237404f40a3533c3c393acc87"
format_stamp: "Formatted at 2026-08-09 00:59:27 on dist-test-slave-07c2"
server_key: "3bf865abd9310a5fe2c71b3efe5a48d6"
server_key_iv: "19253028c62fb5c10c068b8c0375422a"
server_key_version: "encryptionkey@0"
I20260809 00:59:27.721625 7143 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 00:59:27.724841 7159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:27.725996 7143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 00:59:27.726153 7143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "e5eba24237404f40a3533c3c393acc87"
format_stamp: "Formatted at 2026-08-09 00:59:27 on dist-test-slave-07c2"
server_key: "3bf865abd9310a5fe2c71b3efe5a48d6"
server_key_iv: "19253028c62fb5c10c068b8c0375422a"
server_key_version: "encryptionkey@0"
I20260809 00:59:27.726351 7143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:27.748907 7143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:27.753134 7143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:27.753432 7143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:27.754374 7143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:27.755707 7143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:27.755782 7143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:27.755970 7143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:27.756033 7143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:27.789012 7143 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:41225
I20260809 00:59:27.789069 7278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:41225 every 8 connection(s)
I20260809 00:59:27.790290 7143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 00:59:27.792477 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 7143
I20260809 00:59:27.792707 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 00:59:27.793618 4426 external_mini_cluster.cc:1428] Setting key 11d24f81f31b2075c8ed3114d47062fc
Aug 09 00:59:27 dist-test-slave-07c2 krb5kdc[7026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237167, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:27.816141 7281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:27.793045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:38349 (local address 127.4.82.190:35513)
0809 00:59:27.799383 (+ 6338us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:27.799387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:27.799403 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:59:27.799448 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:27.799451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:27.799506 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:27.799591 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:27.801718 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:27.802399 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:27.804682 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:27.805412 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:27.805556 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:27.808978 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:27.808987 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:27.808995 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:27.809040 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:27.812688 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:27.813533 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:27.813541 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:27.813546 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:27.813704 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:27.814123 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:27.814131 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:27.814135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:27.814557 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:27.814742 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:27.815455 (+ 713us) server_negotiation.cc:299] Negotiation successful
0809 00:59:27.815699 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6185,"thread_start_us":6105,"threads_started":1}
I20260809 00:59:27.816911 7282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:27.793509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:35513 (local address 127.4.82.129:38349)
0809 00:59:27.794544 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:27.794598 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:27.796595 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:27.799882 (+ 3287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:27.799895 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:27.800688 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:27.801526 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:27.801554 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:27.802549 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:27.802557 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:27.804519 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:27.804532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:27.805622 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:27.805627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:27.805826 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:27.806527 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:27.806548 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:27.808758 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:27.812825 (+ 4067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:27.812831 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:27.812844 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:27.813385 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:27.813817 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:27.813820 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:27.813822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:27.813993 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:27.814697 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:27.814702 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:27.815015 (+ 313us) client_negotiation.cc:769] Sending connection context
0809 00:59:27.815238 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 00:59:27.815696 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"thread_start_us":568,"threads_started":1}
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
I20260809 00:59:27.818497 7279 heartbeater.cc:347] Connected to a master server at 127.4.82.190:35513
I20260809 00:59:27.818776 7279 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:27.819494 7279 heartbeater.cc:511] Master 127.4.82.190:35513 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
I20260809 00:59:27.821794 7075 ts_manager.cc:194] Registered new tserver with Master: e5eba24237404f40a3533c3c393acc87 (127.4.82.129:41225)
I20260809 00:59:27.823974 7075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:38349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 00:59:27.846756 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:35513
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:28.158335 7287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:28.158650 7287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:28.158825 7287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:28.165864 7287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:28.166003 7287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:28.166188 7287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 00:59:28.175609 7287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:35513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:28.176818 7287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:28.178083 7287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:28.190871 7293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:28.190836 7294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:28.190896 7296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:28.191769 7287 server_base.cc:1034] running on GCE node
I20260809 00:59:28.192437 7287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:28.193482 7287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:28.194712 7287 hybrid_clock.cc:648] HybridClock initialized: now 1786237168194668 us; error 58 us; skew 500 ppm
I20260809 00:59:28.195030 7287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:28 dist-test-slave-07c2 krb5kdc[7026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237168, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:28.197978 7287 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 00:59:28.199476 7287 webserver.cc:460] Webserver started at http://127.4.82.130:40949/ using document root <none> and password file <none>
I20260809 00:59:28.200133 7287 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:28.200279 7287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:28.200592 7287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:28.203455 7287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "462264bd7f0a43f7b13750d9baf1397b"
format_stamp: "Formatted at 2026-08-09 00:59:28 on dist-test-slave-07c2"
server_key: "e9fab1f2641263c81a8a1fdc0c8fe63f"
server_key_iv: "9c085d30bbc3d5443359f367f6b58c66"
server_key_version: "encryptionkey@0"
I20260809 00:59:28.204316 7287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "462264bd7f0a43f7b13750d9baf1397b"
format_stamp: "Formatted at 2026-08-09 00:59:28 on dist-test-slave-07c2"
server_key: "e9fab1f2641263c81a8a1fdc0c8fe63f"
server_key_iv: "9c085d30bbc3d5443359f367f6b58c66"
server_key_version: "encryptionkey@0"
I20260809 00:59:28.209407 7287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 00:59:28.212950 7303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:28.214448 7287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 00:59:28.214630 7287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "462264bd7f0a43f7b13750d9baf1397b"
format_stamp: "Formatted at 2026-08-09 00:59:28 on dist-test-slave-07c2"
server_key: "e9fab1f2641263c81a8a1fdc0c8fe63f"
server_key_iv: "9c085d30bbc3d5443359f367f6b58c66"
server_key_version: "encryptionkey@0"
I20260809 00:59:28.214854 7287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:28.231043 7287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:28.235013 7287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:28.235313 7287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:28.236238 7287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:28.237509 7287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:28.237624 7287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:28.237778 7287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:28.237852 7287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:28.270886 7287 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:34909
I20260809 00:59:28.270941 7434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:34909 every 8 connection(s)
I20260809 00:59:28.272156 7287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 00:59:28.275457 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 7287
I20260809 00:59:28.275681 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 00:59:28.276258 4426 external_mini_cluster.cc:1428] Setting key c3d09bd84e3849e230a035f626a5cc15
Aug 09 00:59:28 dist-test-slave-07c2 krb5kdc[7026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237168, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:28.293569 7281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:28.274659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:52281 (local address 127.4.82.190:35513)
0809 00:59:28.274847 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:28.274852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:28.276214 (+ 1362us) server_negotiation.cc:407] Connection header received
0809 00:59:28.278389 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:28.278396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:28.278477 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:28.278558 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:28.280630 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:28.281339 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:28.283798 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:28.284203 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:28.284294 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:28.287947 (+ 3653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:28.287952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:28.287956 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:28.287987 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:28.290551 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:28.291264 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:28.291269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:28.291272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:28.291441 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:28.291880 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:28.291885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:28.291889 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:28.292156 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:28.292321 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:28.292967 (+ 646us) server_negotiation.cc:299] Negotiation successful
0809 00:59:28.293184 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 00:59:28.294538 7438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:28.275104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:35513 (local address 127.4.82.130:52281)
0809 00:59:28.276010 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:28.276068 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:28.278142 (+ 2074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:28.278842 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:28.278854 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:28.279628 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:28.280452 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:28.280480 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:28.281504 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:28.281512 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:28.283648 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:28.283664 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:28.284415 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:28.284424 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:28.284744 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:28.285480 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:28.285513 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:28.287719 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:28.290692 (+ 2973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:28.290701 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:28.290723 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:28.291120 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:28.291557 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:28.291563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:28.291567 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:28.291757 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:28.292304 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:28.292311 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:28.292615 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 00:59:28.292881 (+ 266us) client_negotiation.cc:240] Negotiation successful
0809 00:59:28.293370 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":361,"threads_started":1}
I20260809 00:59:28.296159 7436 heartbeater.cc:347] Connected to a master server at 127.4.82.190:35513
I20260809 00:59:28.296478 7436 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:28.297155 7436 heartbeater.cc:511] Master 127.4.82.190:35513 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
I20260809 00:59:28.298715 7075 ts_manager.cc:194] Registered new tserver with Master: 462264bd7f0a43f7b13750d9baf1397b (127.4.82.130:34909)
I20260809 00:59:28.299566 7075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:52281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 00:59:28.322844 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:35513
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:28.640686 7441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:28.640941 7441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:28.641034 7441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:28.648064 7441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:28.648173 7441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:28.648293 7441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 00:59:28.657830 7441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:35513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:28.659085 7441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:28.660442 7441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:28.673832 7447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:28.673883 7449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:28.673861 7446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:28.675096 7441 server_base.cc:1034] running on GCE node
I20260809 00:59:28.675822 7441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:28.676738 7441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:28.677970 7441 hybrid_clock.cc:648] HybridClock initialized: now 1786237168677931 us; error 49 us; skew 500 ppm
I20260809 00:59:28.678292 7441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:28 dist-test-slave-07c2 krb5kdc[7026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237168, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:28.681339 7441 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 00:59:28.682708 7441 webserver.cc:460] Webserver started at http://127.4.82.131:35553/ using document root <none> and password file <none>
I20260809 00:59:28.683441 7441 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:28.683589 7441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:28.683892 7441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:28.686662 7441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "4b322facedff4510904eb470ab48ff5b"
format_stamp: "Formatted at 2026-08-09 00:59:28 on dist-test-slave-07c2"
server_key: "6f982b4447df1cefa9b8f686137a9e45"
server_key_iv: "86df3578b5e9e3485d37cb17f9f8ffdb"
server_key_version: "encryptionkey@0"
I20260809 00:59:28.687536 7441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "4b322facedff4510904eb470ab48ff5b"
format_stamp: "Formatted at 2026-08-09 00:59:28 on dist-test-slave-07c2"
server_key: "6f982b4447df1cefa9b8f686137a9e45"
server_key_iv: "86df3578b5e9e3485d37cb17f9f8ffdb"
server_key_version: "encryptionkey@0"
I20260809 00:59:28.692409 7441 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 00:59:28.695681 7456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:28.696856 7441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 00:59:28.697036 7441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "4b322facedff4510904eb470ab48ff5b"
format_stamp: "Formatted at 2026-08-09 00:59:28 on dist-test-slave-07c2"
server_key: "6f982b4447df1cefa9b8f686137a9e45"
server_key_iv: "86df3578b5e9e3485d37cb17f9f8ffdb"
server_key_version: "encryptionkey@0"
I20260809 00:59:28.697238 7441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:28.713315 7441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:28.717005 7441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:28.717290 7441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:28.718175 7441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:28.719532 7441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:28.719635 7441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:28.719782 7441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:28.719856 7441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:28.755270 7441 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:46313
I20260809 00:59:28.755394 7585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:46313 every 8 connection(s)
I20260809 00:59:28.756520 7441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 00:59:28.761313 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 7441
I20260809 00:59:28.761524 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 00:59:28.762079 4426 external_mini_cluster.cc:1428] Setting key 45b2016e6df536c58392dcac3950b46f
Aug 09 00:59:28 dist-test-slave-07c2 krb5kdc[7026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237168, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:28.777674 7281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:28.758871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:56117 (local address 127.4.82.190:35513)
0809 00:59:28.759040 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:28.759045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:28.761178 (+ 2133us) server_negotiation.cc:407] Connection header received
0809 00:59:28.763601 (+ 2423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:28.763605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:28.763693 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:28.763775 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:28.765493 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:28.766249 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:28.768271 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:28.768681 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:28.768775 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:28.772023 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:28.772028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:28.772031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:28.772060 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:28.774516 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:28.775150 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:28.775154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:28.775157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:28.775270 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:28.775798 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:28.775802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:28.775808 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:28.776196 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:28.776332 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:28.777184 (+ 852us) server_negotiation.cc:299] Negotiation successful
0809 00:59:28.777402 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 00:59:28.778546 7588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:28.759340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:35513 (local address 127.4.82.131:56117)
0809 00:59:28.760969 (+ 1629us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:28.761025 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:28.763275 (+ 2250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:28.764054 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:28.764065 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:28.764600 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:28.765340 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:28.765359 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:28.766384 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:28.766388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:28.768147 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:28.768158 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:28.768865 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:28.768869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:28.769046 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:28.769721 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:28.769748 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:28.771849 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:28.774637 (+ 2788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:28.774643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:28.774653 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:28.775046 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:28.775425 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:28.775430 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:28.775433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:28.775651 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:28.776337 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:28.776345 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:28.776753 (+ 408us) client_negotiation.cc:769] Sending connection context
0809 00:59:28.777010 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 00:59:28.777507 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1351,"thread_start_us":1117,"threads_started":1}
I20260809 00:59:28.779872 7586 heartbeater.cc:347] Connected to a master server at 127.4.82.190:35513
I20260809 00:59:28.780141 7586 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:28.780807 7586 heartbeater.cc:511] Master 127.4.82.190:35513 requested a full tablet report, sending...
I20260809 00:59:28.782104 7075 ts_manager.cc:194] Registered new tserver with Master: 4b322facedff4510904eb470ab48ff5b (127.4.82.131:46313)
I20260809 00:59:28.782857 7075 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:56117
I20260809 00:59:28.789461 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 00:59:28.824779 7281 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:59:28.822405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.4.82.190:35513)
0809 00:59:28.822667 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:28.822674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:28.822994 (+ 320us) server_negotiation.cc:407] Connection header received
0809 00:59:28.823136 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:28.823140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:28.823213 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:28.823374 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:28.824515 (+ 1141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44160: BlockingRecv error: recv got EOF from 127.0.0.1:44160 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 00:59:28.827728 7279 heartbeater.cc:503] Master 127.4.82.190:35513 was elected leader, sending a full tablet report...
I20260809 00:59:28.828313 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 7143
W20260809 00:59:28.831698 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 7143: Not authorized: Client connection negotiation failed: client connection to 127.4.82.129:41225: server requires authentication, but client does not have Kerberos credentials available
I20260809 00:59:28.831895 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 7143
I20260809 00:59:28.850322 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 7287
W20260809 00:59:28.852872 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 7287: Not authorized: Client connection negotiation failed: client connection to 127.4.82.130:34909: server requires authentication, but client does not have Kerberos credentials available
I20260809 00:59:28.853034 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 7287
I20260809 00:59:28.869309 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 7441
W20260809 00:59:28.872409 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 7441: Not authorized: Client connection negotiation failed: client connection to 127.4.82.131:46313: server requires authentication, but client does not have Kerberos credentials available
I20260809 00:59:28.872520 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 7441
I20260809 00:59:28.893815 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 7039
I20260809 00:59:28.895233 7109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:29.004503 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 7039
2026-08-09T00:59:29Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3529 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:59:29 dist-test-slave-07c2 krb5kdc[7603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:59:29 dist-test-slave-07c2 krb5kdc[7603](info): set up 2 sockets
Aug 09 00:59:29 dist-test-slave-07c2 krb5kdc[7603](info): commencing operation
krb5kdc: starting...
W20260809 00:59:30.118709 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.005s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 00:59:30 dist-test-slave-07c2 krb5kdc[7603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:59:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 00:59:30.317157 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34177
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:38885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:34177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:30.632583 7617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:30.632861 7617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:30.633057 7617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:30.640061 7617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:30.640182 7617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:30.640285 7617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:30.640388 7617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:30.640475 7617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:30.649649 7617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:38885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:34177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:30.650820 7617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:30.652052 7617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:30.663439 7622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:30.663403 7625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:30.663486 7623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:30.664405 7617 server_base.cc:1034] running on GCE node
I20260809 00:59:30.665189 7617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:30.666563 7617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:30.667790 7617 hybrid_clock.cc:648] HybridClock initialized: now 1786237170667751 us; error 68 us; skew 500 ppm
I20260809 00:59:30.668150 7617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:30 dist-test-slave-07c2 krb5kdc[7603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237170, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:30.675064 7617 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 00:59:30.676476 7617 webserver.cc:460] Webserver started at http://127.4.82.190:32771/ using document root <none> and password file <none>
I20260809 00:59:30.677083 7617 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:30.677177 7617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:30.677428 7617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:30.680347 7617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "8add8ffa442f49f787147039731517c5"
format_stamp: "Formatted at 2026-08-09 00:59:30 on dist-test-slave-07c2"
server_key: "9586e8b49f800f37628752a7d9bf4590"
server_key_iv: "6d33591d8f9acbc95192d8df2d11c68f"
server_key_version: "encryptionkey@0"
I20260809 00:59:30.681070 7617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "8add8ffa442f49f787147039731517c5"
format_stamp: "Formatted at 2026-08-09 00:59:30 on dist-test-slave-07c2"
server_key: "9586e8b49f800f37628752a7d9bf4590"
server_key_iv: "6d33591d8f9acbc95192d8df2d11c68f"
server_key_version: "encryptionkey@0"
I20260809 00:59:30.685912 7617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 00:59:30.689090 7632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:30.690332 7617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 00:59:30.690510 7617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "8add8ffa442f49f787147039731517c5"
format_stamp: "Formatted at 2026-08-09 00:59:30 on dist-test-slave-07c2"
server_key: "9586e8b49f800f37628752a7d9bf4590"
server_key_iv: "6d33591d8f9acbc95192d8df2d11c68f"
server_key_version: "encryptionkey@0"
I20260809 00:59:30.690689 7617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:30.715641 7617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:30.719444 7617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:30.719738 7617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:30.739487 7617 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:34177
I20260809 00:59:30.739511 7692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:34177 every 8 connection(s)
I20260809 00:59:30.740876 7617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:30.744443 7693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:30.746188 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 7617
I20260809 00:59:30.746413 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:30.747231 4426 external_mini_cluster.cc:1428] Setting key bfacc29eb5aa251d48ad788df3956fba
I20260809 00:59:30.759775 7693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5: Bootstrap starting.
Aug 09 00:59:30 dist-test-slave-07c2 krb5kdc[7603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:30.763746 7693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:30.764943 7693 log.cc:824] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:30.767838 7693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5: No bootstrap required, opened a new log
I20260809 00:59:30.771258 7696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:30.749276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53282 (local address 127.4.82.190:34177)
0809 00:59:30.750218 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:30.750234 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:30.750286 (+ 52us) server_negotiation.cc:407] Connection header received
0809 00:59:30.752272 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:30.752298 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:30.752690 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:30.753009 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:30.755056 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:30.756481 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:30.758298 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:30.758972 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:30.759117 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:30.763966 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:30.763973 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:30.763986 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:30.764020 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:30.767106 (+ 3086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:30.767883 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:30.767890 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:30.767896 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:30.768106 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:30.768496 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:30.768502 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:30.768505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:30.769033 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:30.769366 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:30.769834 (+ 468us) server_negotiation.cc:299] Negotiation successful
0809 00:59:30.770273 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":437,"threads_started":1}
I20260809 00:59:30.775249 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:59:30.776562 7693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8add8ffa442f49f787147039731517c5" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34177 } }
I20260809 00:59:30.777055 7693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:30.777196 7693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8add8ffa442f49f787147039731517c5, State: Initialized, Role: FOLLOWER
I20260809 00:59:30.778151 7693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8add8ffa442f49f787147039731517c5" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34177 } }
I20260809 00:59:30.778494 7698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:59:30.778780 7698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8add8ffa442f49f787147039731517c5" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34177 } }
I20260809 00:59:30.778471 7693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:30.779438 7693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:30.779429 7698 leader_election.cc:302] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8add8ffa442f49f787147039731517c5; no voters:
I20260809 00:59:30.779621 7693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:30.780249 7698 leader_election.cc:288] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260809 00:59:30.782097 7693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8add8ffa442f49f787147039731517c5" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34177 } }
I20260809 00:59:30.782353 7693 leader_election.cc:302] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8add8ffa442f49f787147039731517c5; no voters:
I20260809 00:59:30.782397 7699 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 00:59:30.782591 7693 leader_election.cc:288] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:30.782734 7698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:30.784257 7698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [term 1 LEADER]: Becoming Leader. State: Replica: 8add8ffa442f49f787147039731517c5, State: Running, Role: LEADER
I20260809 00:59:30.784750 7698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8add8ffa442f49f787147039731517c5" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34177 } }
I20260809 00:59:30.785126 7693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:30.786942 7699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8add8ffa442f49f787147039731517c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8add8ffa442f49f787147039731517c5" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34177 } } }
I20260809 00:59:30.786998 7700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8add8ffa442f49f787147039731517c5. Latest consensus state: current_term: 1 leader_uuid: "8add8ffa442f49f787147039731517c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8add8ffa442f49f787147039731517c5" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34177 } } }
I20260809 00:59:30.787492 7699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:30.787500 7700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:30.789260 7705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:30.794044 7705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:30.806172 7705 catalog_manager.cc:1347] Generated new cluster ID: 0ebaa404b2c94cf0b5e59719a527cda2
I20260809 00:59:30.806387 7705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:30.833422 7705 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:30.835436 7705 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:30.844362 7705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8add8ffa442f49f787147039731517c5: Generated new TSK 0
I20260809 00:59:30.845475 7705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:59:30.858680 7696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:30.850347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53290 (local address 127.4.82.190:34177)
0809 00:59:30.850534 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:30.850541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:30.851870 (+ 1329us) server_negotiation.cc:407] Connection header received
0809 00:59:30.852083 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:30.852088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:30.852172 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:30.852294 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:30.853352 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:30.854070 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:30.855448 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:30.855893 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:30.856036 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:30.857195 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:30.857200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:30.857203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:30.857862 (+ 659us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:30.857921 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:30.857948 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:30.858078 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:30.858247 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 00:59:30.858352 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 00:59:30.865825 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 7617
I20260809 00:59:30.866998 7685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:30.957512 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 7617
2026-08-09T00:59:30Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1944 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:59:31 dist-test-slave-07c2 krb5kdc[7729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:59:31 dist-test-slave-07c2 krb5kdc[7729](info): set up 2 sockets
Aug 09 00:59:31 dist-test-slave-07c2 krb5kdc[7729](info): commencing operation
krb5kdc: starting...
W20260809 00:59:32.070587 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 00:59:32 dist-test-slave-07c2 krb5kdc[7729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:59:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 00:59:32.274726 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41603
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:41603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:32.587862 7742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:32.588128 7742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:32.588315 7742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:32.595331 7742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:32.595458 7742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:32.595568 7742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:32.595672 7742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:32.595772 7742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:32.605180 7742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:41603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:32.606418 7742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:32.607707 7742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:32.619686 7747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:32.619647 7748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:32.619596 7750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:32.620541 7742 server_base.cc:1034] running on GCE node
I20260809 00:59:32.621402 7742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:32.622941 7742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:32.624176 7742 hybrid_clock.cc:648] HybridClock initialized: now 1786237172624123 us; error 47 us; skew 500 ppm
I20260809 00:59:32.624541 7742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:32 dist-test-slave-07c2 krb5kdc[7729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237172, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:32.627907 7742 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 00:59:32.629372 7742 webserver.cc:460] Webserver started at http://127.4.82.190:34815/ using document root <none> and password file <none>
I20260809 00:59:32.630028 7742 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:32.630134 7742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:32.630394 7742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:32.633574 7742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "a35e1c0680854416896f84b037683ee3"
format_stamp: "Formatted at 2026-08-09 00:59:32 on dist-test-slave-07c2"
server_key: "0a3c4b290c7815d776d180e7130549fa"
server_key_iv: "e3aa38f7e42b6a4156888e49e5d5308c"
server_key_version: "encryptionkey@0"
I20260809 00:59:32.634321 7742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "a35e1c0680854416896f84b037683ee3"
format_stamp: "Formatted at 2026-08-09 00:59:32 on dist-test-slave-07c2"
server_key: "0a3c4b290c7815d776d180e7130549fa"
server_key_iv: "e3aa38f7e42b6a4156888e49e5d5308c"
server_key_version: "encryptionkey@0"
I20260809 00:59:32.639899 7742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 00:59:32.643610 7758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:32.644984 7742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 00:59:32.645186 7742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "a35e1c0680854416896f84b037683ee3"
format_stamp: "Formatted at 2026-08-09 00:59:32 on dist-test-slave-07c2"
server_key: "0a3c4b290c7815d776d180e7130549fa"
server_key_iv: "e3aa38f7e42b6a4156888e49e5d5308c"
server_key_version: "encryptionkey@0"
I20260809 00:59:32.645396 7742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:32.665269 7742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:32.669144 7742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:32.669443 7742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:32.687773 7742 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:41603
I20260809 00:59:32.687842 7814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:41603 every 8 connection(s)
I20260809 00:59:32.689113 7742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:32.693302 7815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:32.693661 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 7742
I20260809 00:59:32.693871 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:32.694835 4426 external_mini_cluster.cc:1428] Setting key 2016610326523ffd5cfbaacd392f63d0
I20260809 00:59:32.708868 7815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3: Bootstrap starting.
Aug 09 00:59:32 dist-test-slave-07c2 krb5kdc[7729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:32.712414 7815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:32.713601 7815 log.cc:824] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:32.716569 7815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3: No bootstrap required, opened a new log
I20260809 00:59:32.720152 7818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:32.697021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34708 (local address 127.4.82.190:41603)
0809 00:59:32.697847 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:32.697861 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:32.697930 (+ 69us) server_negotiation.cc:407] Connection header received
0809 00:59:32.701116 (+ 3186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:32.701137 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:32.701661 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:32.702187 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:32.703859 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:32.705087 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:32.706862 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:32.707567 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:32.707700 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:32.712354 (+ 4654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:32.712362 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:32.712375 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:32.712420 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:32.715912 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:32.716584 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:32.716594 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:32.716606 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:32.716836 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:32.717218 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:32.717223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:32.717227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:32.717738 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:32.718064 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:32.718548 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 00:59:32.719117 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":333,"threads_started":1}
I20260809 00:59:32.724172 7815 raft_consensus.cc:348] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e1c0680854416896f84b037683ee3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41603 } }
I20260809 00:59:32.724519 7815 raft_consensus.cc:374] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:32.724611 7815 raft_consensus.cc:729] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35e1c0680854416896f84b037683ee3, State: Initialized, Role: FOLLOWER
I20260809 00:59:32.725139 7815 consensus_queue.cc:260] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e1c0680854416896f84b037683ee3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41603 } }
I20260809 00:59:32.725368 7815 raft_consensus.cc:388] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:32.725518 7815 raft_consensus.cc:482] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:32.725683 7815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:32.727871 7815 raft_consensus.cc:504] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e1c0680854416896f84b037683ee3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41603 } }
I20260809 00:59:32.728359 7815 leader_election.cc:302] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a35e1c0680854416896f84b037683ee3; no voters:
I20260809 00:59:32.728907 7815 leader_election.cc:288] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:32.729005 7820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:32.729735 7820 raft_consensus.cc:686] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [term 1 LEADER]: Becoming Leader. State: Replica: a35e1c0680854416896f84b037683ee3, State: Running, Role: LEADER
I20260809 00:59:32.730191 7820 consensus_queue.cc:237] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e1c0680854416896f84b037683ee3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41603 } }
I20260809 00:59:32.730877 7815 sys_catalog.cc:564] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:32.732955 7821 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a35e1c0680854416896f84b037683ee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e1c0680854416896f84b037683ee3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41603 } } }
I20260809 00:59:32.733309 7822 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a35e1c0680854416896f84b037683ee3. Latest consensus state: current_term: 1 leader_uuid: "a35e1c0680854416896f84b037683ee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e1c0680854416896f84b037683ee3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41603 } } }
I20260809 00:59:32.733587 7821 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:32.733639 7822 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:32.735850 7828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:32.740948 7828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:32.750262 7828 catalog_manager.cc:1347] Generated new cluster ID: 279bf223c1ad4c86b7216e5c5c4078be
I20260809 00:59:32.750366 7828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:32.762836 7828 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:32.764089 7828 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:32.771135 7828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a35e1c0680854416896f84b037683ee3: Generated new TSK 0
I20260809 00:59:32.772008 7828 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 00:59:32.828650 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41603
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:33.147049 7842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:33.147365 7842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:33.147519 7842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:33.154632 7842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:33.154786 7842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:33.154947 7842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 00:59:33.164170 7842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:33.165339 7842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:33.166563 7842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:33.179417 7850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:33.179476 7847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:33.179378 7848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:33.180348 7842 server_base.cc:1034] running on GCE node
I20260809 00:59:33.180877 7842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:33.181689 7842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:33.182884 7842 hybrid_clock.cc:648] HybridClock initialized: now 1786237173182852 us; error 29 us; skew 500 ppm
I20260809 00:59:33.183185 7842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:33 dist-test-slave-07c2 krb5kdc[7729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237173, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:33.186286 7842 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 00:59:33.187718 7842 webserver.cc:460] Webserver started at http://127.4.82.129:36369/ using document root <none> and password file <none>
I20260809 00:59:33.188345 7842 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:33.188488 7842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:33.188804 7842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:33.191574 7842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "a8b9f7c5c17042f18d894f9eaa4e9ea1"
format_stamp: "Formatted at 2026-08-09 00:59:33 on dist-test-slave-07c2"
server_key: "ae684dcbea56f6913470879ba14f3dc8"
server_key_iv: "79c2f5d617b74b9ddf6d214993cda5b4"
server_key_version: "encryptionkey@0"
I20260809 00:59:33.192340 7842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "a8b9f7c5c17042f18d894f9eaa4e9ea1"
format_stamp: "Formatted at 2026-08-09 00:59:33 on dist-test-slave-07c2"
server_key: "ae684dcbea56f6913470879ba14f3dc8"
server_key_iv: "79c2f5d617b74b9ddf6d214993cda5b4"
server_key_version: "encryptionkey@0"
I20260809 00:59:33.197188 7842 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 00:59:33.200502 7858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:33.201705 7842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 00:59:33.201872 7842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "a8b9f7c5c17042f18d894f9eaa4e9ea1"
format_stamp: "Formatted at 2026-08-09 00:59:33 on dist-test-slave-07c2"
server_key: "ae684dcbea56f6913470879ba14f3dc8"
server_key_iv: "79c2f5d617b74b9ddf6d214993cda5b4"
server_key_version: "encryptionkey@0"
I20260809 00:59:33.202066 7842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:33.231549 7842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:33.235736 7842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:33.236027 7842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:33.236882 7842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:33.238191 7842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:33.238305 7842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:33.238451 7842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:33.238534 7842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:33.272253 7842 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:44211
I20260809 00:59:33.272336 7980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:44211 every 8 connection(s)
I20260809 00:59:33.273617 7842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 00:59:33.277930 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 7842
I20260809 00:59:33.278159 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 00:59:33.279009 4426 external_mini_cluster.cc:1428] Setting key 844267e1c07cdcbb1e5aadb18b6517e2
Aug 09 00:59:33 dist-test-slave-07c2 krb5kdc[7729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237173, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:33.295662 7983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:33.276202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:45781 (local address 127.4.82.190:41603)
0809 00:59:33.276638 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:33.276643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:33.277697 (+ 1054us) server_negotiation.cc:407] Connection header received
0809 00:59:33.280205 (+ 2508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:33.280210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:33.280284 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:33.280415 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:33.282538 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:33.283253 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:33.284833 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:33.285246 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:33.285389 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:33.288832 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:33.288840 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:33.288848 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:33.288894 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:33.292464 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:33.293147 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:33.293156 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:33.293162 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:33.293318 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:33.293721 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:33.293728 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:33.293734 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:33.294104 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:33.294255 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:33.294911 (+ 656us) server_negotiation.cc:299] Negotiation successful
0809 00:59:33.295153 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":217,"threads_started":1}
I20260809 00:59:33.296262 7984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:33.276789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41603 (local address 127.4.82.129:45781)
0809 00:59:33.277511 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:33.277562 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:33.279909 (+ 2347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:33.281164 (+ 1255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:33.281177 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:33.281848 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:33.282369 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:33.282385 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:33.283465 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:33.283471 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:33.284675 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:33.284687 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:33.285471 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:33.285481 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:33.285740 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:33.286415 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:33.286437 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:33.288612 (+ 2175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:33.292612 (+ 4000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:33.292618 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:33.292628 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:33.293023 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:33.293430 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:33.293433 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:33.293435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:33.293602 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:33.294224 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:33.294230 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:33.294516 (+ 286us) client_negotiation.cc:769] Sending connection context
0809 00:59:33.294743 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 00:59:33.295092 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":288,"threads_started":1}
I20260809 00:59:33.297848 7981 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41603
I20260809 00:59:33.298146 7981 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:33.298846 7981 heartbeater.cc:511] Master 127.4.82.190:41603 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
I20260809 00:59:33.301067 7776 ts_manager.cc:194] Registered new tserver with Master: a8b9f7c5c17042f18d894f9eaa4e9ea1 (127.4.82.129:44211)
I20260809 00:59:33.302457 7776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:45781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 00:59:33.325727 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41603
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:33.643858 7987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:33.644165 7987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:33.644361 7987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:33.651646 7987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:33.651782 7987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:33.651983 7987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 00:59:33.661062 7987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:33.662346 7987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:33.663689 7987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:33.676487 7993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:33.676447 7992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:33.676537 7995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:33.677278 7987 server_base.cc:1034] running on GCE node
I20260809 00:59:33.677860 7987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:33.678759 7987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:33.680008 7987 hybrid_clock.cc:648] HybridClock initialized: now 1786237173679967 us; error 27 us; skew 500 ppm
I20260809 00:59:33.680344 7987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:33 dist-test-slave-07c2 krb5kdc[7729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237173, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:33.683610 7987 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 00:59:33.684922 7987 webserver.cc:460] Webserver started at http://127.4.82.130:33395/ using document root <none> and password file <none>
I20260809 00:59:33.685591 7987 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:33.685732 7987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:33.686041 7987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:33.689659 7987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "2e1726af603d42b09ca06529b5156927"
format_stamp: "Formatted at 2026-08-09 00:59:33 on dist-test-slave-07c2"
server_key: "92f1cfe0b1f520b1495520b2884893c6"
server_key_iv: "cab98244e6927cf09e2ff1cc414dccb1"
server_key_version: "encryptionkey@0"
I20260809 00:59:33.690591 7987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "2e1726af603d42b09ca06529b5156927"
format_stamp: "Formatted at 2026-08-09 00:59:33 on dist-test-slave-07c2"
server_key: "92f1cfe0b1f520b1495520b2884893c6"
server_key_iv: "cab98244e6927cf09e2ff1cc414dccb1"
server_key_version: "encryptionkey@0"
I20260809 00:59:33.696851 7987 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260809 00:59:33.700935 8002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:33.702404 7987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 00:59:33.702557 7987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "2e1726af603d42b09ca06529b5156927"
format_stamp: "Formatted at 2026-08-09 00:59:33 on dist-test-slave-07c2"
server_key: "92f1cfe0b1f520b1495520b2884893c6"
server_key_iv: "cab98244e6927cf09e2ff1cc414dccb1"
server_key_version: "encryptionkey@0"
I20260809 00:59:33.702800 7987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:33.732019 7987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:33.735795 7987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:33.736040 7987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:33.736876 7987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:33.738088 7987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:33.738158 7987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:33.738243 7987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:33.738294 7987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:33.772456 7987 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:45801
I20260809 00:59:33.772543 8133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:45801 every 8 connection(s)
I20260809 00:59:33.773686 7987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 00:59:33.783646 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 7987
I20260809 00:59:33.783852 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 00:59:33.784431 4426 external_mini_cluster.cc:1428] Setting key b8dbe5ca9bdf0a9b637f0a98a262b9ec
Aug 09 00:59:33 dist-test-slave-07c2 krb5kdc[7729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237173, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:33.794353 7983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:33.776265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:34065 (local address 127.4.82.190:41603)
0809 00:59:33.776435 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:33.776439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:33.777600 (+ 1161us) server_negotiation.cc:407] Connection header received
0809 00:59:33.779442 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:33.779448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:33.779548 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:33.779668 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:33.781136 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:33.782089 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:33.784186 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:33.784650 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:33.784761 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:33.788672 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:33.788677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:33.788680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:33.788713 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:33.791368 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:33.792083 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:33.792089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:33.792092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:33.792225 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:33.792661 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:33.792671 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:33.792673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:33.792936 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:33.793091 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:33.793764 (+ 673us) server_negotiation.cc:299] Negotiation successful
0809 00:59:33.793990 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 00:59:33.795245 8136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:33.776672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41603 (local address 127.4.82.130:34065)
0809 00:59:33.777398 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:33.777436 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:33.779106 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:33.779903 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:33.779911 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:33.780416 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:33.780956 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:33.780972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:33.782277 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:33.782285 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:33.784026 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:33.784043 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:33.784889 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:33.784897 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:33.785205 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:33.785951 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:33.785975 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:33.788439 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:33.791508 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:33.791517 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:33.791532 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:33.791952 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:33.792339 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:33.792344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:33.792348 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:33.792537 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:33.793067 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:33.793074 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:33.793406 (+ 332us) client_negotiation.cc:769] Sending connection context
0809 00:59:33.793663 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 00:59:33.794178 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":307,"threads_started":1}
I20260809 00:59:33.796676 8134 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41603
I20260809 00:59:33.796998 8134 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:33.797915 8134 heartbeater.cc:511] Master 127.4.82.190:41603 requested a full tablet report, sending...
I20260809 00:59:33.799527 7776 ts_manager.cc:194] Registered new tserver with Master: 2e1726af603d42b09ca06529b5156927 (127.4.82.130:45801)
I20260809 00:59:33.800321 7776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:34065
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 00:59:33.832254 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41603
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:34.153331 8139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:34.153585 8139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:34.153785 8139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:34.160861 8139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:34.160967 8139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:34.161175 8139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 00:59:34.170724 8139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:34.172024 8139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:34.173301 8139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:34.185930 8147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:34.185985 8145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:34.186009 8144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:34.186832 8139 server_base.cc:1034] running on GCE node
I20260809 00:59:34.187500 8139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:34.188344 8139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:34.189534 8139 hybrid_clock.cc:648] HybridClock initialized: now 1786237174189503 us; error 30 us; skew 500 ppm
I20260809 00:59:34.189862 8139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:34 dist-test-slave-07c2 krb5kdc[7729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237174, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:34.192899 8139 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 00:59:34.194190 8139 webserver.cc:460] Webserver started at http://127.4.82.131:43353/ using document root <none> and password file <none>
I20260809 00:59:34.194844 8139 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:34.194979 8139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:34.195348 8139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:34.198040 8139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "7cbfa5215ad848a9b16180aec447e98c"
format_stamp: "Formatted at 2026-08-09 00:59:34 on dist-test-slave-07c2"
server_key: "3d25c1c430004aca47561ad9d7d86e95"
server_key_iv: "c88c32aa99de2550f4e183b8776be264"
server_key_version: "encryptionkey@0"
I20260809 00:59:34.198812 8139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "7cbfa5215ad848a9b16180aec447e98c"
format_stamp: "Formatted at 2026-08-09 00:59:34 on dist-test-slave-07c2"
server_key: "3d25c1c430004aca47561ad9d7d86e95"
server_key_iv: "c88c32aa99de2550f4e183b8776be264"
server_key_version: "encryptionkey@0"
I20260809 00:59:34.203833 8139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 00:59:34.207406 8155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:34.208799 8139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 00:59:34.208974 8139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "7cbfa5215ad848a9b16180aec447e98c"
format_stamp: "Formatted at 2026-08-09 00:59:34 on dist-test-slave-07c2"
server_key: "3d25c1c430004aca47561ad9d7d86e95"
server_key_iv: "c88c32aa99de2550f4e183b8776be264"
server_key_version: "encryptionkey@0"
I20260809 00:59:34.209146 8139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:34.223012 8139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:34.226666 8139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:34.226907 8139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:34.227823 8139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:34.229043 8139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:34.229113 8139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:34.229194 8139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:34.229251 8139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:34.262719 8139 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:35901
I20260809 00:59:34.262758 8276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:35901 every 8 connection(s)
I20260809 00:59:34.263983 8139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 00:59:34.271378 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 8139
I20260809 00:59:34.271564 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 00:59:34.272141 4426 external_mini_cluster.cc:1428] Setting key 170febee1a2a60e06d7c30f3fdf244bf
Aug 09 00:59:34 dist-test-slave-07c2 krb5kdc[7729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237174, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:34.283917 7983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:34.266486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:47135 (local address 127.4.82.190:41603)
0809 00:59:34.266666 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:34.266670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:34.267763 (+ 1093us) server_negotiation.cc:407] Connection header received
0809 00:59:34.269557 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:34.269563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:34.269682 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:34.269809 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:34.271491 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:34.272637 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:34.274794 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:34.275375 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:34.275521 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:34.278931 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:34.278936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:34.278939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:34.278971 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:34.281435 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:34.282058 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:34.282061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:34.282064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:34.282183 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:34.282553 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:34.282556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:34.282558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:34.282796 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:34.282904 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:34.283533 (+ 629us) server_negotiation.cc:299] Negotiation successful
0809 00:59:34.283686 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 00:59:34.284757 8279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:34.266869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41603 (local address 127.4.82.131:47135)
0809 00:59:34.267603 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:34.267645 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:34.269292 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:34.270036 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:34.270043 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:34.270543 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:34.271250 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:34.271276 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:34.272831 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:34.272834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:34.274630 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:34.274641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:34.275643 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:34.275652 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:34.275894 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:34.276594 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:34.276616 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:34.278746 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:34.281562 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:34.281568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:34.281577 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:34.281960 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:34.282289 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:34.282292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:34.282294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:34.282462 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:34.282900 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:34.282905 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:34.283167 (+ 262us) client_negotiation.cc:769] Sending connection context
0809 00:59:34.283441 (+ 274us) client_negotiation.cc:240] Negotiation successful
0809 00:59:34.283783 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":293,"threads_started":1}
I20260809 00:59:34.286105 8277 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41603
I20260809 00:59:34.286432 8277 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:34.287228 8277 heartbeater.cc:511] Master 127.4.82.190:41603 requested a full tablet report, sending...
I20260809 00:59:34.288705 7776 ts_manager.cc:194] Registered new tserver with Master: 7cbfa5215ad848a9b16180aec447e98c (127.4.82.131:35901)
I20260809 00:59:34.289484 7776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:47135
I20260809 00:59:34.299755 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:59:34.305882 7981 heartbeater.cc:503] Master 127.4.82.190:41603 was elected leader, sending a full tablet report...
Aug 09 00:59:34 dist-test-slave-07c2 krb5kdc[7729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237174, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 00:59:34 dist-test-slave-07c2 krb5kdc[7729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237174, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:34.349781 7983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:34.336581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34712 (local address 127.4.82.190:41603)
0809 00:59:34.336854 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:34.336862 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:34.337169 (+ 307us) server_negotiation.cc:407] Connection header received
0809 00:59:34.337303 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:34.337306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:34.337373 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:34.337453 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:34.338706 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:34.339482 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:34.340669 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:34.341052 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:34.341140 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:34.344868 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:34.344873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:34.344876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:34.344905 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:34.347622 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:34.348128 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:34.348132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:34.348134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:34.348250 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:34.348552 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:34.348555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:34.348557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:34.348910 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:34.349145 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:34.349337 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 00:59:34.349482 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260809 00:59:34.350349 7776 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34712
I20260809 00:59:34.352057 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 7842
Aug 09 00:59:34 dist-test-slave-07c2 krb5kdc[7729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237174, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 00:59:34.367594 8288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:34.353128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40888 (local address 127.4.82.129:44211)
0809 00:59:34.353815 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:34.353822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:34.353845 (+ 23us) server_negotiation.cc:407] Connection header received
0809 00:59:34.353894 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:34.353900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:34.354100 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:34.354218 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:34.355247 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:34.356249 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:34.357339 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:34.357877 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:34.357968 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:34.361474 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:34.361484 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:34.361496 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:34.361526 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:34.365001 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:34.365758 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:34.365766 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:34.365768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:34.365908 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:34.366286 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:34.366288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:34.366290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:34.366578 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:34.366791 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:34.366995 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 00:59:34.367168 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":377,"threads_started":1}
W20260809 00:59:34.368925 7975 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40888
W20260809 00:59:34.369953 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 7842: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 00:59:34.370159 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 7842
I20260809 00:59:34.389941 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 7987
Aug 09 00:59:34 dist-test-slave-07c2 krb5kdc[7729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237174, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 00:59:34.404973 8289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:34.390754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41844 (local address 127.4.82.130:45801)
0809 00:59:34.391374 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:34.391380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:34.391406 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:59:34.391463 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:34.391469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:34.391663 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:34.391805 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:34.393065 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:34.394038 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:34.395370 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:34.395903 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:34.396000 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:34.399969 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:34.399979 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:34.399991 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:34.400022 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:34.402608 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:34.403174 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:34.403180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:34.403182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:34.403406 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:34.403722 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:34.403725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:34.403727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:34.404020 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:34.404240 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:34.404472 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 00:59:34.404664 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":297,"threads_started":1}
W20260809 00:59:34.406209 8129 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41844
W20260809 00:59:34.407225 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 7987: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 00:59:34.407449 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 7987
I20260809 00:59:34.424918 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 8139
Aug 09 00:59:34 dist-test-slave-07c2 krb5kdc[7729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237174, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 00:59:34.440762 8290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:34.425710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57008 (local address 127.4.82.131:35901)
0809 00:59:34.426332 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:34.426339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:34.426366 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:59:34.426422 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:34.426428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:34.426625 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:34.426748 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:34.428117 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:34.429203 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:34.430329 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:34.430851 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:34.430949 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:34.435419 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:34.435431 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:34.435444 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:34.435474 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:34.438223 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:34.438824 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:34.438830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:34.438832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:34.438944 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:34.439268 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:34.439271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:34.439273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:34.439638 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:34.439876 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:34.440147 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 00:59:34.440350 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":318,"threads_started":1}
W20260809 00:59:34.441927 8271 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57008
W20260809 00:59:34.442992 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 8139: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 00:59:34.443171 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 8139
I20260809 00:59:34.460639 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 7742
I20260809 00:59:34.461848 7810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:34.565181 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 7742
2026-08-09T00:59:34Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3616 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:59:34 dist-test-slave-07c2 krb5kdc[8294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:59:34 dist-test-slave-07c2 krb5kdc[8294](info): set up 2 sockets
Aug 09 00:59:34 dist-test-slave-07c2 krb5kdc[8294](info): commencing operation
krb5kdc: starting...
W20260809 00:59:35.682731 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 00:59:35 dist-test-slave-07c2 krb5kdc[8294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:59:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 00:59:35.880865 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:32855
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:32855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:36.207078 8308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:36.207360 8308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:36.207544 8308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:36.214476 8308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:36.214603 8308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:36.214722 8308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:36.214831 8308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:36.214916 8308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:36.224656 8308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:32855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:32855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:36.225858 8308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:36.227097 8308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:36.239179 8316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:36.239271 8313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:36.239354 8314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:36.240182 8308 server_base.cc:1034] running on GCE node
I20260809 00:59:36.240935 8308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:36.242236 8308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:36.243690 8308 hybrid_clock.cc:648] HybridClock initialized: now 1786237176243618 us; error 47 us; skew 500 ppm
I20260809 00:59:36.244056 8308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:36 dist-test-slave-07c2 krb5kdc[8294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237176, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:36.251631 8308 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 00:59:36.253083 8308 webserver.cc:460] Webserver started at http://127.4.82.190:40453/ using document root <none> and password file <none>
I20260809 00:59:36.253734 8308 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:36.253907 8308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:36.254287 8308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:36.257294 8308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "bcea582789024dcdbf2272c37cc8f514"
format_stamp: "Formatted at 2026-08-09 00:59:36 on dist-test-slave-07c2"
server_key: "373516082314e6e6e625db6e349babb0"
server_key_iv: "d865a5fb86a4a0db071c151f229682d7"
server_key_version: "encryptionkey@0"
I20260809 00:59:36.258158 8308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "bcea582789024dcdbf2272c37cc8f514"
format_stamp: "Formatted at 2026-08-09 00:59:36 on dist-test-slave-07c2"
server_key: "373516082314e6e6e625db6e349babb0"
server_key_iv: "d865a5fb86a4a0db071c151f229682d7"
server_key_version: "encryptionkey@0"
I20260809 00:59:36.264053 8308 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260809 00:59:36.267359 8323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:36.268500 8308 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 00:59:36.268699 8308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "bcea582789024dcdbf2272c37cc8f514"
format_stamp: "Formatted at 2026-08-09 00:59:36 on dist-test-slave-07c2"
server_key: "373516082314e6e6e625db6e349babb0"
server_key_iv: "d865a5fb86a4a0db071c151f229682d7"
server_key_version: "encryptionkey@0"
I20260809 00:59:36.268899 8308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:36.304147 8308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:36.307770 8308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:36.308087 8308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:36.326071 8308 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:32855
I20260809 00:59:36.326107 8376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:32855 every 8 connection(s)
I20260809 00:59:36.327417 8308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:36.328645 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 8308
I20260809 00:59:36.328914 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:36.329743 4426 external_mini_cluster.cc:1428] Setting key 1d1f3c22093ecccccc0ff1441eb1819a
I20260809 00:59:36.332005 8377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 00:59:36 dist-test-slave-07c2 krb5kdc[8294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:36.345325 8377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514: Bootstrap starting.
I20260809 00:59:36.348656 8377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:36.349898 8377 log.cc:824] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:36.353257 8377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514: No bootstrap required, opened a new log
I20260809 00:59:36.353528 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:36.331816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41466 (local address 127.4.82.190:32855)
0809 00:59:36.332597 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:36.332611 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:36.332658 (+ 47us) server_negotiation.cc:407] Connection header received
0809 00:59:36.334839 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:36.334865 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:36.335350 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:36.335806 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:36.337129 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:36.339138 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:36.340499 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:36.341505 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:36.341690 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:36.345883 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:36.345890 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:36.345903 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:36.345941 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:36.348960 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:36.349759 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:36.349765 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:36.349770 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:36.349920 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:36.350329 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:36.350332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:36.350334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:36.351115 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:36.351461 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:36.351919 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 00:59:36.352308 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":348,"threads_started":1}
I20260809 00:59:36.361071 8377 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcea582789024dcdbf2272c37cc8f514" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 32855 } }
I20260809 00:59:36.361378 8377 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:36.361449 8377 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcea582789024dcdbf2272c37cc8f514, State: Initialized, Role: FOLLOWER
I20260809 00:59:36.361950 8377 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcea582789024dcdbf2272c37cc8f514" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 32855 } }
I20260809 00:59:36.362143 8377 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:36.362242 8377 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:36.362365 8377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:36.364550 8377 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcea582789024dcdbf2272c37cc8f514" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 32855 } }
I20260809 00:59:36.364955 8377 leader_election.cc:302] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcea582789024dcdbf2272c37cc8f514; no voters:
I20260809 00:59:36.365476 8377 leader_election.cc:288] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:36.365561 8383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:36.366264 8383 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [term 1 LEADER]: Becoming Leader. State: Replica: bcea582789024dcdbf2272c37cc8f514, State: Running, Role: LEADER
I20260809 00:59:36.366776 8383 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcea582789024dcdbf2272c37cc8f514" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 32855 } }
I20260809 00:59:36.367705 8377 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:36.369623 8384 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcea582789024dcdbf2272c37cc8f514" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcea582789024dcdbf2272c37cc8f514" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 32855 } } }
I20260809 00:59:36.370011 8384 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:36.375257 8385 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcea582789024dcdbf2272c37cc8f514. Latest consensus state: current_term: 1 leader_uuid: "bcea582789024dcdbf2272c37cc8f514" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcea582789024dcdbf2272c37cc8f514" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 32855 } } }
I20260809 00:59:36.375583 8385 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:36.376240 8389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:36.382241 8389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:36.390353 8389 catalog_manager.cc:1347] Generated new cluster ID: 511831cdbbcd4bfb8950f4885aada57f
I20260809 00:59:36.390496 8389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:36.402657 8389 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:36.404374 8389 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:36.414347 8389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcea582789024dcdbf2272c37cc8f514: Generated new TSK 0
I20260809 00:59:36.415097 8389 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 00:59:36.471869 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:32855
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:36.786566 8405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:36.786821 8405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:36.786976 8405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:36.793898 8405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:36.794013 8405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:36.794193 8405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 00:59:36.803241 8405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:32855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:36.804427 8405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:36.805691 8405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:36.818472 8410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:36.818406 8411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:36.818405 8414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:36.819203 8405 server_base.cc:1034] running on GCE node
I20260809 00:59:36.819815 8405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:36.820711 8405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:36.821934 8405 hybrid_clock.cc:648] HybridClock initialized: now 1786237176821892 us; error 59 us; skew 500 ppm
I20260809 00:59:36.822242 8405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:36 dist-test-slave-07c2 krb5kdc[8294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237176, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:36.825289 8405 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 00:59:36.826674 8405 webserver.cc:460] Webserver started at http://127.4.82.129:41399/ using document root <none> and password file <none>
I20260809 00:59:36.827368 8405 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:36.827509 8405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:36.827813 8405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:36.830564 8405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "9e8fbb35d93e4995a4ec885838faba08"
format_stamp: "Formatted at 2026-08-09 00:59:36 on dist-test-slave-07c2"
server_key: "d64944b9b21c8aa8c4255a9c2082914d"
server_key_iv: "08f8f8edffe28fc575decdaf606fde42"
server_key_version: "encryptionkey@0"
I20260809 00:59:36.831420 8405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "9e8fbb35d93e4995a4ec885838faba08"
format_stamp: "Formatted at 2026-08-09 00:59:36 on dist-test-slave-07c2"
server_key: "d64944b9b21c8aa8c4255a9c2082914d"
server_key_iv: "08f8f8edffe28fc575decdaf606fde42"
server_key_version: "encryptionkey@0"
I20260809 00:59:36.836619 8405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 00:59:36.839955 8423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:36.841176 8405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 00:59:36.841346 8405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "9e8fbb35d93e4995a4ec885838faba08"
format_stamp: "Formatted at 2026-08-09 00:59:36 on dist-test-slave-07c2"
server_key: "d64944b9b21c8aa8c4255a9c2082914d"
server_key_iv: "08f8f8edffe28fc575decdaf606fde42"
server_key_version: "encryptionkey@0"
I20260809 00:59:36.841562 8405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:36.858486 8405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:36.862385 8405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:36.862681 8405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:36.863765 8405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:36.865141 8405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:36.865257 8405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:36.865417 8405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:36.865489 8405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:36.899578 8405 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:43985
I20260809 00:59:36.899667 8539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:43985 every 8 connection(s)
I20260809 00:59:36.900890 8405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 00:59:36.910151 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 8405
I20260809 00:59:36.910370 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 00:59:36.911005 4426 external_mini_cluster.cc:1428] Setting key fc636e939836a082ee0f70b60aa8bb67
Aug 09 00:59:36 dist-test-slave-07c2 krb5kdc[8294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237176, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:36.922338 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:36.903474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:35169 (local address 127.4.82.190:32855)
0809 00:59:36.903921 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:36.903926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:36.904834 (+ 908us) server_negotiation.cc:407] Connection header received
0809 00:59:36.906580 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:36.906586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:36.906669 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:36.906788 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:36.908348 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:36.909160 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:36.911034 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:36.911580 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:36.911724 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:36.915696 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:36.915706 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:36.915721 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:36.915764 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:36.919246 (+ 3482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:36.919978 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:36.919986 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:36.919992 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:36.920158 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:36.920692 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:36.920695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:36.920697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:36.920957 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:36.921067 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:36.921918 (+ 851us) server_negotiation.cc:299] Negotiation successful
0809 00:59:36.922082 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":222,"threads_started":1}
I20260809 00:59:36.923790 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:36.903996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:32855 (local address 127.4.82.129:35169)
0809 00:59:36.904682 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:36.904718 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:36.906311 (+ 1593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:36.907016 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:36.907024 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:36.907658 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:36.908175 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:36.908191 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:36.909323 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:36.909327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:36.910818 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:36.910830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:36.911865 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:36.911872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:36.912148 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:36.912873 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:36.912910 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:36.915490 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:36.919432 (+ 3942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:36.919438 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:36.919448 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:36.919846 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:36.920272 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:36.920276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:36.920278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:36.920570 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:36.921092 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:36.921102 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:36.921537 (+ 435us) client_negotiation.cc:769] Sending connection context
0809 00:59:36.921842 (+ 305us) client_negotiation.cc:240] Negotiation successful
0809 00:59:36.922394 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":274,"threads_started":1}
I20260809 00:59:36.925254 8540 heartbeater.cc:347] Connected to a master server at 127.4.82.190:32855
I20260809 00:59:36.925535 8540 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:36.926213 8540 heartbeater.cc:511] Master 127.4.82.190:32855 requested a full tablet report, sending...
I20260809 00:59:36.928191 8341 ts_manager.cc:194] Registered new tserver with Master: 9e8fbb35d93e4995a4ec885838faba08 (127.4.82.129:43985)
I20260809 00:59:36.929719 8341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:35169
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 00:59:36.958114 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:32855
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:37.273798 8547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:37.274097 8547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:37.274276 8547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:37.281281 8547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:37.281421 8547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:37.281598 8547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 00:59:37.290719 8547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:32855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:37.291981 8547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:37.293267 8547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:37.305780 8552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:37.305858 8555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:37.305827 8553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:37.306890 8547 server_base.cc:1034] running on GCE node
I20260809 00:59:37.307504 8547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:37.308389 8547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:37.309610 8547 hybrid_clock.cc:648] HybridClock initialized: now 1786237177309563 us; error 34 us; skew 500 ppm
I20260809 00:59:37.309904 8547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:37 dist-test-slave-07c2 krb5kdc[8294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237177, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:37.312920 8547 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 00:59:37.314258 8547 webserver.cc:460] Webserver started at http://127.4.82.130:35475/ using document root <none> and password file <none>
I20260809 00:59:37.314829 8547 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:37.314930 8547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:37.315187 8547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:37.318452 8547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "4d21f944d00b4faab6bd13d2a9a1221a"
format_stamp: "Formatted at 2026-08-09 00:59:37 on dist-test-slave-07c2"
server_key: "8b481650e2ccc2a20658d29b72956646"
server_key_iv: "598475243ec25dd9beb95b2728b9f2e0"
server_key_version: "encryptionkey@0"
I20260809 00:59:37.319149 8547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "4d21f944d00b4faab6bd13d2a9a1221a"
format_stamp: "Formatted at 2026-08-09 00:59:37 on dist-test-slave-07c2"
server_key: "8b481650e2ccc2a20658d29b72956646"
server_key_iv: "598475243ec25dd9beb95b2728b9f2e0"
server_key_version: "encryptionkey@0"
I20260809 00:59:37.323877 8547 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 00:59:37.327234 8562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:37.328601 8547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 00:59:37.328763 8547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "4d21f944d00b4faab6bd13d2a9a1221a"
format_stamp: "Formatted at 2026-08-09 00:59:37 on dist-test-slave-07c2"
server_key: "8b481650e2ccc2a20658d29b72956646"
server_key_iv: "598475243ec25dd9beb95b2728b9f2e0"
server_key_version: "encryptionkey@0"
I20260809 00:59:37.328941 8547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:37.343828 8547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:37.347908 8547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:37.348124 8547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:37.348958 8547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:37.350157 8547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:37.350231 8547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:37.350311 8547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:37.350354 8547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:37.383069 8547 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:34407
I20260809 00:59:37.383104 8680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:34407 every 8 connection(s)
I20260809 00:59:37.384315 8547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 00:59:37.394598 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 8547
I20260809 00:59:37.394837 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 00:59:37.395467 4426 external_mini_cluster.cc:1428] Setting key a1623c7ac8e6e8882c72f8b158bf4c6c
Aug 09 00:59:37 dist-test-slave-07c2 krb5kdc[8294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237177, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:37.403473 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:37.386734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:59135 (local address 127.4.82.190:32855)
0809 00:59:37.386909 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:37.386913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:37.388052 (+ 1139us) server_negotiation.cc:407] Connection header received
0809 00:59:37.389849 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:37.389855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:37.389942 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:37.390052 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:37.391524 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.392270 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.393716 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.394120 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.394255 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:37.397925 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:37.397930 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:37.397933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:37.397962 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:37.400475 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:37.401181 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:37.401186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:37.401192 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:37.401323 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:37.401761 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:37.401771 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:37.401773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:37.402013 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:37.402199 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:37.402905 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 00:59:37.403118 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 00:59:37.404371 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:37.387156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:32855 (local address 127.4.82.130:59135)
0809 00:59:37.387888 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:37.387921 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:37.389581 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:37.390280 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:37.390287 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:37.390757 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:37.391360 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:37.391380 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.392425 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.392437 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:37.393571 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:37.393583 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.394338 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.394345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:37.394564 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:37.395275 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:37.395357 (+ 82us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:37.397721 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:37.400611 (+ 2890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:37.400620 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:37.400632 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:37.401050 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:37.401446 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:37.401451 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:37.401455 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:37.401637 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:37.402156 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:37.402172 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:37.402534 (+ 362us) client_negotiation.cc:769] Sending connection context
0809 00:59:37.402808 (+ 274us) client_negotiation.cc:240] Negotiation successful
0809 00:59:37.403309 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":318,"threads_started":1}
I20260809 00:59:37.405766 8681 heartbeater.cc:347] Connected to a master server at 127.4.82.190:32855
I20260809 00:59:37.406062 8681 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:37.406864 8681 heartbeater.cc:511] Master 127.4.82.190:32855 requested a full tablet report, sending...
I20260809 00:59:37.408416 8341 ts_manager.cc:194] Registered new tserver with Master: 4d21f944d00b4faab6bd13d2a9a1221a (127.4.82.130:34407)
I20260809 00:59:37.409269 8341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:59135
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 00:59:37.441847 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:32855
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:37.766410 8686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:37.766660 8686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:37.766816 8686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:37.773947 8686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:37.774061 8686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:37.774253 8686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 00:59:37.783492 8686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:32855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:37.784555 8686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:37.785766 8686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:37.797871 8691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:37.797915 8692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:37.797876 8694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:37.799072 8686 server_base.cc:1034] running on GCE node
I20260809 00:59:37.799746 8686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:37.800593 8686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:37.801824 8686 hybrid_clock.cc:648] HybridClock initialized: now 1786237177801789 us; error 48 us; skew 500 ppm
I20260809 00:59:37.802127 8686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:37 dist-test-slave-07c2 krb5kdc[8294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237177, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:37.805145 8686 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 00:59:37.806458 8686 webserver.cc:460] Webserver started at http://127.4.82.131:44381/ using document root <none> and password file <none>
I20260809 00:59:37.807101 8686 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:37.807258 8686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:37.807647 8686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:37.810508 8686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "9604c953912448d189704fbac005ce19"
format_stamp: "Formatted at 2026-08-09 00:59:37 on dist-test-slave-07c2"
server_key: "88a0ad7f71d2f910b06ffffda876c97f"
server_key_iv: "f3ae7a7ad5288b73ca829ef0d09977fa"
server_key_version: "encryptionkey@0"
I20260809 00:59:37.811286 8686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "9604c953912448d189704fbac005ce19"
format_stamp: "Formatted at 2026-08-09 00:59:37 on dist-test-slave-07c2"
server_key: "88a0ad7f71d2f910b06ffffda876c97f"
server_key_iv: "f3ae7a7ad5288b73ca829ef0d09977fa"
server_key_version: "encryptionkey@0"
I20260809 00:59:37.816365 8686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 00:59:37.819664 8701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:37.820900 8686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 00:59:37.821072 8686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "9604c953912448d189704fbac005ce19"
format_stamp: "Formatted at 2026-08-09 00:59:37 on dist-test-slave-07c2"
server_key: "88a0ad7f71d2f910b06ffffda876c97f"
server_key_iv: "f3ae7a7ad5288b73ca829ef0d09977fa"
server_key_version: "encryptionkey@0"
I20260809 00:59:37.821266 8686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:37.842582 8686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:37.846791 8686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:37.847065 8686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:37.848058 8686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:37.849362 8686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:37.849465 8686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:37.849606 8686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:37.849684 8686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:37.882807 8686 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:40179
I20260809 00:59:37.882887 8820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:40179 every 8 connection(s)
I20260809 00:59:37.884092 8686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 00:59:37.890038 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 8686
I20260809 00:59:37.890247 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 00:59:37.890643 4426 external_mini_cluster.cc:1428] Setting key a28a87555bf8d33a9a45d5d7825ce355
Aug 09 00:59:37 dist-test-slave-07c2 krb5kdc[8294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237177, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:37.903334 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:37.886525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:48411 (local address 127.4.82.190:32855)
0809 00:59:37.886705 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:37.886710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:37.887862 (+ 1152us) server_negotiation.cc:407] Connection header received
0809 00:59:37.889668 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:37.889674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:37.889772 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:37.889888 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:37.891532 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.892324 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.893832 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.894232 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.894367 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:37.897649 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:37.897654 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:37.897657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:37.897693 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:37.900202 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:37.901044 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:37.901050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:37.901053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:37.901217 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:37.901630 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:37.901635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:37.901638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:37.902011 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:37.902165 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:37.902759 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 00:59:37.902979 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 00:59:37.904182 8823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:37.886945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:32855 (local address 127.4.82.131:48411)
0809 00:59:37.887698 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:37.887740 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:37.889403 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:37.890138 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:37.890151 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:37.890759 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:37.891336 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:37.891376 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.892485 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.892490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:37.893681 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:37.893692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.894453 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.894462 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:37.894713 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:37.895420 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:37.895441 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:37.897442 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:37.900337 (+ 2895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:37.900345 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:37.900357 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:37.900891 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:37.901333 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:37.901336 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:37.901339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:37.901505 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:37.902136 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:37.902141 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:37.902400 (+ 259us) client_negotiation.cc:769] Sending connection context
0809 00:59:37.902598 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 00:59:37.903024 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":320,"threads_started":1}
I20260809 00:59:37.905499 8821 heartbeater.cc:347] Connected to a master server at 127.4.82.190:32855
I20260809 00:59:37.905753 8821 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:37.906339 8821 heartbeater.cc:511] Master 127.4.82.190:32855 requested a full tablet report, sending...
I20260809 00:59:37.907680 8341 ts_manager.cc:194] Registered new tserver with Master: 9604c953912448d189704fbac005ce19 (127.4.82.131:40179)
I20260809 00:59:37.908419 8341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:48411
I20260809 00:59:37.917430 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:59:37.932535 8540 heartbeater.cc:503] Master 127.4.82.190:32855 was elected leader, sending a full tablet report...
Aug 09 00:59:37 dist-test-slave-07c2 krb5kdc[8294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:59:37 dist-test-slave-07c2 krb5kdc[8294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 00:59:37.968317 8829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:37.953420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.4.82.129:43985)
0809 00:59:37.954239 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:37.954246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:37.954271 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:59:37.954328 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:37.954335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:37.954537 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:37.954662 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:37.956278 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.957276 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.958483 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.959026 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.959125 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:37.962756 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:37.962766 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:37.962779 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:37.962812 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:37.965440 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:37.966108 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:37.966115 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:37.966117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:37.966242 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:37.966570 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:37.966573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:37.966575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:37.966885 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:37.967176 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:37.967568 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 00:59:37.967853 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":425,"threads_started":1}
Aug 09 00:59:37 dist-test-slave-07c2 krb5kdc[8294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:37.985136 8542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:37.972275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41478 (local address 127.4.82.190:32855)
0809 00:59:37.972490 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:37.972496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:37.972713 (+ 217us) server_negotiation.cc:407] Connection header received
0809 00:59:37.972911 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:37.972915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:37.972976 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:37.973081 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:37.974317 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.975032 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.976358 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.976774 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.976909 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:37.980682 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:37.980690 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:37.980694 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:37.980736 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:37.983081 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:37.983686 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:37.983689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:37.983691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:37.983814 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:37.984116 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:37.984119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:37.984121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:37.984397 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:37.984560 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:37.984784 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 00:59:37.984923 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 00:59:37.985702 8341 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41478
I20260809 00:59:37.986626 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 8405
I20260809 00:59:37.998335 8829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:37.987421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.4.82.129:43985)
0809 00:59:37.987558 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:37.987562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:37.987919 (+ 357us) server_negotiation.cc:407] Connection header received
0809 00:59:37.988019 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:37.988041 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:37.988094 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:37.988173 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:37.989243 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.989879 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.990973 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:37.991450 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:37.991548 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:37.993656 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:37.993673 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:37.993676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:37.993704 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:37.996354 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:37.996900 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:37.996904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:37.996906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:37.997018 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:37.997313 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:37.997316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:37.997318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:37.997528 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:37.997703 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:37.997955 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 00:59:37.998120 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 00:59:37.998873 8534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:38.080380 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 8405
I20260809 00:59:38.097790 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 8547
Aug 09 00:59:38 dist-test-slave-07c2 krb5kdc[8294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 00:59:38.113332 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:38.098651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.4.82.130:34407)
0809 00:59:38.099169 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:38.099176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:38.099201 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:59:38.099261 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:38.099267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:38.099533 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:38.099670 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:38.100749 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:38.101764 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:38.102990 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:38.103654 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:38.103759 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:38.107824 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:38.107835 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:38.107847 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:38.107877 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:38.110830 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:38.111541 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:38.111548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:38.111550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:38.111682 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:38.111984 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:38.111987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:38.111989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:38.112298 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:38.112525 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:38.112750 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 00:59:38.112938 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":265,"threads_started":1}
I20260809 00:59:38.114494 8676 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:38.196236 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 8547
I20260809 00:59:38.214783 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 8686
Aug 09 00:59:38 dist-test-slave-07c2 krb5kdc[8294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 00:59:38.230463 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:38.215834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51534 (local address 127.4.82.131:40179)
0809 00:59:38.216433 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:38.216440 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:38.216467 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:59:38.216525 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:38.216531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:38.216746 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:38.216868 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:38.218127 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:38.219135 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:38.220452 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:38.220994 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:38.221094 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:38.225386 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:38.225396 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:38.225409 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:38.225445 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:38.228176 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:38.228785 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:38.228792 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:38.228794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:38.228920 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:38.229224 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:38.229227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:38.229229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:38.229536 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:38.229774 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:38.229998 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 00:59:38.230183 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":297,"threads_started":1}
I20260809 00:59:38.231750 8816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:38.314008 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 8686
I20260809 00:59:38.332336 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 8308
I20260809 00:59:38.333505 8372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:38.436300 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 8308
2026-08-09T00:59:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3871 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:59:38 dist-test-slave-07c2 krb5kdc[8842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:59:38 dist-test-slave-07c2 krb5kdc[8842](info): set up 2 sockets
Aug 09 00:59:38 dist-test-slave-07c2 krb5kdc[8842](info): commencing operation
krb5kdc: starting...
W20260809 00:59:39.550899 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 00:59:39 dist-test-slave-07c2 krb5kdc[8842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:59:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 00:59:39.758642 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:37517
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:37517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:40.073925 8855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:40.074255 8855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:40.074436 8855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:40.081307 8855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:40.081436 8855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:40.081544 8855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:40.081653 8855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:40.081756 8855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:40.091251 8855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:37517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:37517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:40.092446 8855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:40.093750 8855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:40.104951 8861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:40.105118 8863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:40.105811 8855 server_base.cc:1034] running on GCE node
W20260809 00:59:40.106056 8860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:40.106877 8855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:40.108299 8855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:40.109532 8855 hybrid_clock.cc:648] HybridClock initialized: now 1786237180109502 us; error 41 us; skew 500 ppm
I20260809 00:59:40.109867 8855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:40 dist-test-slave-07c2 krb5kdc[8842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237180, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:40.112907 8855 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 00:59:40.114060 8855 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:40.114205 8855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:40.114526 8855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:40.117524 8855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "4b244245ce7748e4bbfb4fdc84318872"
format_stamp: "Formatted at 2026-08-09 00:59:40 on dist-test-slave-07c2"
server_key: "192f06c096888d370bd679f1ba50e3cd"
server_key_iv: "bbe91945892cf458a3f5ee02d2e0194c"
server_key_version: "encryptionkey@0"
I20260809 00:59:40.118314 8855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "4b244245ce7748e4bbfb4fdc84318872"
format_stamp: "Formatted at 2026-08-09 00:59:40 on dist-test-slave-07c2"
server_key: "192f06c096888d370bd679f1ba50e3cd"
server_key_iv: "bbe91945892cf458a3f5ee02d2e0194c"
server_key_version: "encryptionkey@0"
I20260809 00:59:40.123051 8855 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260809 00:59:40.126223 8869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:40.127493 8855 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:59:40.127667 8855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "4b244245ce7748e4bbfb4fdc84318872"
format_stamp: "Formatted at 2026-08-09 00:59:40 on dist-test-slave-07c2"
server_key: "192f06c096888d370bd679f1ba50e3cd"
server_key_iv: "bbe91945892cf458a3f5ee02d2e0194c"
server_key_version: "encryptionkey@0"
I20260809 00:59:40.127876 8855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:40.144613 8855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:40.148391 8855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:40.148675 8855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:40.167238 8855 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:37517
I20260809 00:59:40.167261 8924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:37517 every 8 connection(s)
I20260809 00:59:40.168174 8855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:40.171650 8925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:40.177424 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 8855
I20260809 00:59:40.177666 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:40.178297 4426 external_mini_cluster.cc:1428] Setting key 33052ceabca2a71d21fc53db907ac9e7
I20260809 00:59:40.182600 8925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872: Bootstrap starting.
I20260809 00:59:40.186383 8925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:40.187692 8925 log.cc:824] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:40.190701 8925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872: No bootstrap required, opened a new log
Aug 09 00:59:40 dist-test-slave-07c2 krb5kdc[8842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:40.200035 8925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b244245ce7748e4bbfb4fdc84318872" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37517 } }
I20260809 00:59:40.200505 8925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:40.200620 8925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b244245ce7748e4bbfb4fdc84318872, State: Initialized, Role: FOLLOWER
I20260809 00:59:40.201306 8925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b244245ce7748e4bbfb4fdc84318872" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37517 } }
I20260809 00:59:40.201617 8925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:40.201772 8925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:40.201997 8925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:40.203557 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:40.180325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43154 (local address 127.4.82.190:37517)
0809 00:59:40.181136 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:40.181155 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:40.181205 (+ 50us) server_negotiation.cc:407] Connection header received
0809 00:59:40.183679 (+ 2474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:40.183703 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:40.184160 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:40.184615 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:40.185979 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:40.188065 (+ 2086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:40.189666 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:40.190671 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:40.190860 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:40.194710 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:40.194721 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:40.194738 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:40.194783 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:40.198787 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:40.199581 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:40.199589 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:40.199599 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:40.199838 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:40.200202 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:40.200206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:40.200209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:40.200979 (+ 770us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:40.201263 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:40.201693 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 00:59:40.202147 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":315,"threads_started":1}
I20260809 00:59:40.205050 8925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b244245ce7748e4bbfb4fdc84318872" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37517 } }
I20260809 00:59:40.205685 8925 leader_election.cc:302] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b244245ce7748e4bbfb4fdc84318872; no voters:
I20260809 00:59:40.206337 8925 leader_election.cc:288] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:40.206482 8930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:40.207278 8930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [term 1 LEADER]: Becoming Leader. State: Replica: 4b244245ce7748e4bbfb4fdc84318872, State: Running, Role: LEADER
I20260809 00:59:40.207825 8930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b244245ce7748e4bbfb4fdc84318872" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37517 } }
I20260809 00:59:40.208453 8925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:40.210633 8932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b244245ce7748e4bbfb4fdc84318872. Latest consensus state: current_term: 1 leader_uuid: "4b244245ce7748e4bbfb4fdc84318872" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b244245ce7748e4bbfb4fdc84318872" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37517 } } }
I20260809 00:59:40.210826 8931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b244245ce7748e4bbfb4fdc84318872" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b244245ce7748e4bbfb4fdc84318872" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37517 } } }
I20260809 00:59:40.211014 8932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:40.211114 8931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:40.213196 8939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:40.217974 8939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:40.226310 8939 catalog_manager.cc:1347] Generated new cluster ID: 84627ed7b9e44f9eabf0605ff0371a00
I20260809 00:59:40.226485 8939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:40.245353 8939 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:40.246599 8939 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:40.255149 8939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872: Generated new TSK 0
I20260809 00:59:40.255927 8939 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 00:59:40.306319 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37517
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:40.629891 8953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:40.630239 8953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:40.630420 8953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:40.637810 8953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:40.637966 8953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:40.638163 8953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 00:59:40.647782 8953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:40.649070 8953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:40.650354 8953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:40.663110 8959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:40.663055 8958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:40.663702 8961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:40.663965 8953 server_base.cc:1034] running on GCE node
I20260809 00:59:40.664551 8953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:40.665454 8953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:40.666692 8953 hybrid_clock.cc:648] HybridClock initialized: now 1786237180666652 us; error 46 us; skew 500 ppm
I20260809 00:59:40.667007 8953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:40 dist-test-slave-07c2 krb5kdc[8842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237180, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:40.670125 8953 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 00:59:40.671226 8953 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:40.671378 8953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:40.671691 8953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:40.674520 8953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "162b73a90804441aad9a9c74be7856e4"
format_stamp: "Formatted at 2026-08-09 00:59:40 on dist-test-slave-07c2"
server_key: "7e6192179fd701c47fb4eddaee330ecf"
server_key_iv: "a3e8b67db0b40c159327e3694dc12e7b"
server_key_version: "encryptionkey@0"
I20260809 00:59:40.675395 8953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "162b73a90804441aad9a9c74be7856e4"
format_stamp: "Formatted at 2026-08-09 00:59:40 on dist-test-slave-07c2"
server_key: "7e6192179fd701c47fb4eddaee330ecf"
server_key_iv: "a3e8b67db0b40c159327e3694dc12e7b"
server_key_version: "encryptionkey@0"
I20260809 00:59:40.680207 8953 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 00:59:40.683423 8967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:40.684650 8953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 00:59:40.684823 8953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "162b73a90804441aad9a9c74be7856e4"
format_stamp: "Formatted at 2026-08-09 00:59:40 on dist-test-slave-07c2"
server_key: "7e6192179fd701c47fb4eddaee330ecf"
server_key_iv: "a3e8b67db0b40c159327e3694dc12e7b"
server_key_version: "encryptionkey@0"
I20260809 00:59:40.685041 8953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:40.699924 8953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:40.704172 8953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:40.704438 8953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:40.705355 8953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:40.706666 8953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:40.706790 8953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:40.706928 8953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:40.707021 8953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:40.740387 8953 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:40211
I20260809 00:59:40.740463 9084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:40211 every 8 connection(s)
I20260809 00:59:40.741251 8953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 00:59:40.747891 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 8953
I20260809 00:59:40.748117 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 00:59:40.748767 4426 external_mini_cluster.cc:1428] Setting key 544bb83db5fd2bee559ec7f0c41924e5
Aug 09 00:59:40 dist-test-slave-07c2 krb5kdc[8842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237180, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:40.761601 9088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:40.743945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:37517 (local address 127.4.82.129:60481)
0809 00:59:40.744614 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:40.744656 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:40.746288 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:40.747008 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:40.747016 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:40.747578 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:40.748167 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:40.748211 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:40.749317 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:40.749322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:40.750837 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:40.750850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:40.751680 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:40.751689 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:40.751902 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:40.752577 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:40.752600 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:40.754824 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:40.757961 (+ 3137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:40.757969 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:40.757983 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:40.758406 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:40.758802 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:40.758807 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:40.758810 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:40.758988 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:40.759648 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:40.759655 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:40.759983 (+ 328us) client_negotiation.cc:769] Sending connection context
0809 00:59:40.760265 (+ 282us) client_negotiation.cc:240] Negotiation successful
0809 00:59:40.760645 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":268,"threads_started":1}
I20260809 00:59:40.762457 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:40.743537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:60481 (local address 127.4.82.190:37517)
0809 00:59:40.743987 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:40.743991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:40.744792 (+ 801us) server_negotiation.cc:407] Connection header received
0809 00:59:40.746555 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:40.746562 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:40.746663 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:40.746787 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:40.748370 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:40.749170 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:40.750992 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:40.751450 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:40.751585 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:40.755041 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:40.755045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:40.755049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:40.755078 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:40.757829 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:40.758536 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:40.758542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:40.758545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:40.758674 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:40.759104 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:40.759111 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:40.759114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:40.759520 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:40.759678 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:40.762051 (+ 2373us) server_negotiation.cc:299] Negotiation successful
0809 00:59:40.762219 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":247,"threads_started":1}
I20260809 00:59:40.763638 9085 heartbeater.cc:347] Connected to a master server at 127.4.82.190:37517
I20260809 00:59:40.763962 9085 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:40.764609 9085 heartbeater.cc:511] Master 127.4.82.190:37517 requested a full tablet report, sending...
I20260809 00:59:40.766562 8887 ts_manager.cc:194] Registered new tserver with Master: 162b73a90804441aad9a9c74be7856e4 (127.4.82.129:40211)
I20260809 00:59:40.768194 8887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:60481
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 00:59:40.794658 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37517
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:41.112964 9091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:41.113308 9091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:41.113442 9091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:41.120716 9091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:41.120817 9091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:41.120986 9091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 00:59:41.130518 9091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:41.131858 9091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:41.133204 9091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:41.145610 9096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:41.145569 9099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:41.145664 9097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:41.151672 9091 server_base.cc:1034] running on GCE node
I20260809 00:59:41.152259 9091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:41.153170 9091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:41.154407 9091 hybrid_clock.cc:648] HybridClock initialized: now 1786237181154361 us; error 54 us; skew 500 ppm
I20260809 00:59:41.154708 9091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:41 dist-test-slave-07c2 krb5kdc[8842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237181, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:41.157861 9091 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 00:59:41.159010 9091 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:41.159148 9091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:41.159541 9091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:41.162411 9091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "a44ede2103e44887993dd92a6bed0e80"
format_stamp: "Formatted at 2026-08-09 00:59:41 on dist-test-slave-07c2"
server_key: "1a10355af0bef0f2c131bac4299d9f96"
server_key_iv: "56a2ca0582606acfdea91c6688413a28"
server_key_version: "encryptionkey@0"
I20260809 00:59:41.163329 9091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "a44ede2103e44887993dd92a6bed0e80"
format_stamp: "Formatted at 2026-08-09 00:59:41 on dist-test-slave-07c2"
server_key: "1a10355af0bef0f2c131bac4299d9f96"
server_key_iv: "56a2ca0582606acfdea91c6688413a28"
server_key_version: "encryptionkey@0"
I20260809 00:59:41.168238 9091 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 00:59:41.171562 9105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:41.172889 9091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 00:59:41.173062 9091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "a44ede2103e44887993dd92a6bed0e80"
format_stamp: "Formatted at 2026-08-09 00:59:41 on dist-test-slave-07c2"
server_key: "1a10355af0bef0f2c131bac4299d9f96"
server_key_iv: "56a2ca0582606acfdea91c6688413a28"
server_key_version: "encryptionkey@0"
I20260809 00:59:41.173276 9091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:41.212456 9091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:41.216744 9091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:41.217015 9091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:41.217990 9091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:41.219401 9091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:41.219514 9091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:41.219648 9091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:41.219744 9091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:41.252938 9091 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:39129
I20260809 00:59:41.253023 9220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:39129 every 8 connection(s)
I20260809 00:59:41.253805 9091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 00:59:41.254097 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 9091
I20260809 00:59:41.254311 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 00:59:41.254927 4426 external_mini_cluster.cc:1428] Setting key 303a1f70da94dad8eb1b90ee03b7b5bc
Aug 09 00:59:41 dist-test-slave-07c2 krb5kdc[8842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237181, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
I20260809 00:59:41.277805 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.256542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:43329 (local address 127.4.82.190:37517)
0809 00:59:41.256731 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:41.256736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:41.258073 (+ 1337us) server_negotiation.cc:407] Connection header received
0809 00:59:41.259976 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:41.259982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:41.260071 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:41.260177 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:41.261747 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.262621 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.264299 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.264701 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.264798 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.271897 (+ 7099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:41.271902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:41.271905 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:41.271935 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:41.274602 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:41.275363 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:41.275368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:41.275370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:41.275495 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:41.275940 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:41.275945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:41.275947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:41.276202 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:41.276385 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:41.277190 (+ 805us) server_negotiation.cc:299] Negotiation successful
0809 00:59:41.277417 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 00:59:41.278759 9223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.257095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:37517 (local address 127.4.82.130:43329)
0809 00:59:41.257914 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:41.257972 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:41.259724 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:41.260399 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:41.260406 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:41.260894 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:41.261580 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.261598 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.262792 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.262795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.264074 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.264119 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.264923 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.264931 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.265257 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.268930 (+ 3673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:41.268971 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:41.271686 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:41.274744 (+ 3058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:41.274753 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:41.274768 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:41.275206 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:41.275617 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:41.275622 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:41.275626 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:41.275804 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:41.276361 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:41.276369 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:41.276814 (+ 445us) client_negotiation.cc:769] Sending connection context
0809 00:59:41.277088 (+ 274us) client_negotiation.cc:240] Negotiation successful
0809 00:59:41.277626 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":338,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
I20260809 00:59:41.280345 9221 heartbeater.cc:347] Connected to a master server at 127.4.82.190:37517
I20260809 00:59:41.280653 9221 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:41.281317 9221 heartbeater.cc:511] Master 127.4.82.190:37517 requested a full tablet report, sending...
I20260809 00:59:41.282712 8887 ts_manager.cc:194] Registered new tserver with Master: a44ede2103e44887993dd92a6bed0e80 (127.4.82.130:39129)
I20260809 00:59:41.283565 8887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:43329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 00:59:41.304270 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37517
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:41.630712 9226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:41.631026 9226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:41.631202 9226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:41.638398 9226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:41.638495 9226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:41.638710 9226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 00:59:41.648077 9226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:41.649300 9226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:41.650525 9226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:41.663327 9231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:41.663417 9232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:41.663460 9234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:41.664572 9226 server_base.cc:1034] running on GCE node
I20260809 00:59:41.665143 9226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:41.666043 9226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:41.667351 9226 hybrid_clock.cc:648] HybridClock initialized: now 1786237181667246 us; error 59 us; skew 500 ppm
I20260809 00:59:41.667691 9226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:41 dist-test-slave-07c2 krb5kdc[8842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237181, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:41.670748 9226 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 00:59:41.671998 9226 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:41.672135 9226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:41.672451 9226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:41.675371 9226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "0c61c1aad8bb4187a31e0751826576ce"
format_stamp: "Formatted at 2026-08-09 00:59:41 on dist-test-slave-07c2"
server_key: "9443129d24a619866773f8dcfd4ea654"
server_key_iv: "287944565c29f5d526a3d2c9c07f6414"
server_key_version: "encryptionkey@0"
I20260809 00:59:41.676249 9226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "0c61c1aad8bb4187a31e0751826576ce"
format_stamp: "Formatted at 2026-08-09 00:59:41 on dist-test-slave-07c2"
server_key: "9443129d24a619866773f8dcfd4ea654"
server_key_iv: "287944565c29f5d526a3d2c9c07f6414"
server_key_version: "encryptionkey@0"
I20260809 00:59:41.681099 9226 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 00:59:41.684458 9240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:41.685595 9226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 00:59:41.685778 9226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "0c61c1aad8bb4187a31e0751826576ce"
format_stamp: "Formatted at 2026-08-09 00:59:41 on dist-test-slave-07c2"
server_key: "9443129d24a619866773f8dcfd4ea654"
server_key_iv: "287944565c29f5d526a3d2c9c07f6414"
server_key_version: "encryptionkey@0"
I20260809 00:59:41.685992 9226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:41.702342 9226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:41.705991 9226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:41.706353 9226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:41.707268 9226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:41.708654 9226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:41.708778 9226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:41.708902 9226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:41.709005 9226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:41.745759 9226 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:45379
I20260809 00:59:41.745810 9360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:45379 every 8 connection(s)
I20260809 00:59:41.746562 9226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 00:59:41.754070 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 9226
I20260809 00:59:41.754271 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 00:59:41.754853 4426 external_mini_cluster.cc:1428] Setting key be6938b70e8c33ac4d59d2f6d7648c7e
Aug 09 00:59:41 dist-test-slave-07c2 krb5kdc[8842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237181, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:41.766296 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.748880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:35305 (local address 127.4.82.190:37517)
0809 00:59:41.749093 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:41.749099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:41.750157 (+ 1058us) server_negotiation.cc:407] Connection header received
0809 00:59:41.751926 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:41.751930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:41.752010 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:41.752102 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:41.753445 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.754269 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.756527 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.757183 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.757310 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.760641 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:41.760646 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:41.760649 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:41.760679 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:41.763359 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:41.764055 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:41.764061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:41.764064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:41.764208 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:41.764629 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:41.764634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:41.764639 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:41.764903 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:41.765062 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:41.765693 (+ 631us) server_negotiation.cc:299] Negotiation successful
0809 00:59:41.765914 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 00:59:41.767156 9363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.749312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:37517 (local address 127.4.82.131:35305)
0809 00:59:41.749998 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:41.750038 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:41.751693 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:41.752278 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:41.752285 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:41.752782 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:41.753297 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.753318 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.754460 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.754465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.756387 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.756402 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.757385 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.757390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.757562 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.758294 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:41.758316 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:41.760455 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:41.763488 (+ 3033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:41.763496 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:41.763516 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:41.763926 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:41.764322 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:41.764327 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:41.764332 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:41.764506 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:41.765035 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:41.765043 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:41.765345 (+ 302us) client_negotiation.cc:769] Sending connection context
0809 00:59:41.765598 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 00:59:41.766077 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":287,"threads_started":1}
I20260809 00:59:41.768462 9361 heartbeater.cc:347] Connected to a master server at 127.4.82.190:37517
I20260809 00:59:41.768778 9361 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:41.769392 9361 heartbeater.cc:511] Master 127.4.82.190:37517 requested a full tablet report, sending...
I20260809 00:59:41.770759 8887 ts_manager.cc:194] Registered new tserver with Master: 0c61c1aad8bb4187a31e0751826576ce (127.4.82.131:45379)
I20260809 00:59:41.770992 9085 heartbeater.cc:503] Master 127.4.82.190:37517 was elected leader, sending a full tablet report...
I20260809 00:59:41.771785 8887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:35305
I20260809 00:59:41.782037 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:59:41.799436 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.786456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43164 (local address 127.4.82.190:37517)
0809 00:59:41.786617 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:41.786625 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:41.787111 (+ 486us) server_negotiation.cc:407] Connection header received
0809 00:59:41.787327 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:41.787335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:41.787411 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:41.787516 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:41.789089 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.789900 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.791206 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.791705 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.791847 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.794168 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:41.794175 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:41.794179 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:41.794218 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:41.796604 (+ 2386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:41.797256 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:41.797263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:41.797266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:41.797414 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:41.797804 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:41.797810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:41.797814 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:41.798125 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:41.798368 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:41.798855 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 00:59:41.799078 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 00:59:41.803329 8886 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43164:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:59:41.810454 8886 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:59:41.849128 9373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.822118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:39129 (local address 127.0.0.1:53590)
0809 00:59:41.824706 (+ 2588us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:41.824767 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:41.824973 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:41.841034 (+ 16061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:41.841043 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:41.841072 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:41.841695 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.841708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.843781 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.843787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.846318 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.846333 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.847980 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.847987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.848198 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.848258 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 00:59:41.848396 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 00:59:41.848636 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2243,"mutex_wait_us":173,"thread_start_us":2046,"threads_started":1}
I20260809 00:59:41.849640 9376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.823131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56066 (local address 127.4.82.129:40211)
0809 00:59:41.840701 (+ 17570us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:41.840714 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:41.840751 (+ 37us) server_negotiation.cc:407] Connection header received
0809 00:59:41.840851 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:41.840857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:41.841072 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:41.841328 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:41.842117 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.843682 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.847019 (+ 3337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.848493 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.848665 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.848868 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:41.849128 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 00:59:41.849328 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17319,"thread_start_us":17256,"threads_started":1}
I20260809 00:59:41.849879 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.825037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53494 (local address 127.4.82.131:45379)
0809 00:59:41.840217 (+ 15180us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:41.840226 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:41.841190 (+ 964us) server_negotiation.cc:407] Connection header received
0809 00:59:41.841460 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:41.841469 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:41.841682 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:41.841879 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:41.842647 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.843802 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.847040 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.848096 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.848243 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.848464 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:41.849366 (+ 902us) server_negotiation.cc:299] Negotiation successful
0809 00:59:41.849527 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15008,"thread_start_us":15171,"threads_started":1}
I20260809 00:59:41.850062 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.822196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53590 (local address 127.4.82.130:39129)
0809 00:59:41.840144 (+ 17948us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:41.840153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:41.840185 (+ 32us) server_negotiation.cc:407] Connection header received
0809 00:59:41.840304 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:41.840311 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:41.840592 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:41.840817 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:41.841852 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.843606 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.846483 (+ 2877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.847776 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.847916 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.848130 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:41.849518 (+ 1388us) server_negotiation.cc:299] Negotiation successful
0809 00:59:41.849695 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17754,"thread_start_us":17611,"threads_started":1}
I20260809 00:59:41.849242 9377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.824300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:45379 (local address 127.0.0.1:53494)
0809 00:59:41.841049 (+ 16749us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:41.841068 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:41.841230 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:41.842165 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:41.842168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:41.842186 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:41.842466 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.842473 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.843986 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.843994 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.846853 (+ 2859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.846868 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.848321 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.848328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.848504 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.848563 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 00:59:41.848651 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 00:59:41.848795 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16598,"mutex_wait_us":170,"thread_start_us":16452,"threads_started":1}
I20260809 00:59:41.849606 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.822394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:40211 (local address 127.0.0.1:56066)
0809 00:59:41.833950 (+ 11556us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:41.833975 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:41.834126 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:41.841336 (+ 7210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:41.841340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:41.841370 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:41.841909 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.841925 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.843868 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.843874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.846858 (+ 2984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.846868 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.848744 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.848750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.848929 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.848968 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 00:59:41.849077 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 00:59:41.849255 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11376,"mutex_wait_us":292,"thread_start_us":12108,"threads_started":1}
I20260809 00:59:41.853710 9290 tablet_service.cc:1467] Processing CreateTablet for tablet ec01010b264d4ef2999fb0ab5f732901 (DEFAULT_TABLE table=test-table [id=b9cafbd828fd4a169eecf2558be74104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:41.853860 9016 tablet_service.cc:1467] Processing CreateTablet for tablet ec01010b264d4ef2999fb0ab5f732901 (DEFAULT_TABLE table=test-table [id=b9cafbd828fd4a169eecf2558be74104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:41.853849 9153 tablet_service.cc:1467] Processing CreateTablet for tablet ec01010b264d4ef2999fb0ab5f732901 (DEFAULT_TABLE table=test-table [id=b9cafbd828fd4a169eecf2558be74104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:41.854915 9290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec01010b264d4ef2999fb0ab5f732901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:41.855131 9016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec01010b264d4ef2999fb0ab5f732901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:41.855124 9153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec01010b264d4ef2999fb0ab5f732901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:41.866290 9379 tablet_bootstrap.cc:492] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce: Bootstrap starting.
I20260809 00:59:41.869338 9379 tablet_bootstrap.cc:654] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:41.870316 9379 log.cc:824] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:41.875315 9380 tablet_bootstrap.cc:492] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4: Bootstrap starting.
I20260809 00:59:41.877015 9381 tablet_bootstrap.cc:492] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80: Bootstrap starting.
I20260809 00:59:41.878685 9380 tablet_bootstrap.cc:654] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:41.879894 9380 log.cc:824] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:41.880851 9381 tablet_bootstrap.cc:654] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:41.882099 9379 tablet_bootstrap.cc:492] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce: No bootstrap required, opened a new log
I20260809 00:59:41.882056 9381 log.cc:824] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:41.882397 9380 tablet_bootstrap.cc:492] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4: No bootstrap required, opened a new log
I20260809 00:59:41.882380 9379 ts_tablet_manager.cc:1397] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.007s
I20260809 00:59:41.882632 9380 ts_tablet_manager.cc:1397] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 00:59:41.889804 9381 tablet_bootstrap.cc:492] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80: No bootstrap required, opened a new log
I20260809 00:59:41.890060 9381 ts_tablet_manager.cc:1397] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.008s
I20260809 00:59:41.890614 9380 raft_consensus.cc:348] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ede2103e44887993dd92a6bed0e80" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39129 } } peers { permanent_uuid: "162b73a90804441aad9a9c74be7856e4" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 40211 } } peers { permanent_uuid: "0c61c1aad8bb4187a31e0751826576ce" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 45379 } }
I20260809 00:59:41.890993 9380 raft_consensus.cc:374] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:41.891090 9380 raft_consensus.cc:729] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 162b73a90804441aad9a9c74be7856e4, State: Initialized, Role: FOLLOWER
I20260809 00:59:41.891752 9380 consensus_queue.cc:260] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ede2103e44887993dd92a6bed0e80" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39129 } } peers { permanent_uuid: "162b73a90804441aad9a9c74be7856e4" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 40211 } } peers { permanent_uuid: "0c61c1aad8bb4187a31e0751826576ce" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 45379 } }
I20260809 00:59:41.893244 9380 ts_tablet_manager.cc:1428] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260809 00:59:41.895364 9379 raft_consensus.cc:348] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ede2103e44887993dd92a6bed0e80" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39129 } } peers { permanent_uuid: "162b73a90804441aad9a9c74be7856e4" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 40211 } } peers { permanent_uuid: "0c61c1aad8bb4187a31e0751826576ce" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 45379 } }
I20260809 00:59:41.895774 9379 raft_consensus.cc:374] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:41.895874 9379 raft_consensus.cc:729] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c61c1aad8bb4187a31e0751826576ce, State: Initialized, Role: FOLLOWER
I20260809 00:59:41.897119 9379 consensus_queue.cc:260] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ede2103e44887993dd92a6bed0e80" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39129 } } peers { permanent_uuid: "162b73a90804441aad9a9c74be7856e4" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 40211 } } peers { permanent_uuid: "0c61c1aad8bb4187a31e0751826576ce" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 45379 } }
I20260809 00:59:41.898335 9361 heartbeater.cc:503] Master 127.4.82.190:37517 was elected leader, sending a full tablet report...
I20260809 00:59:41.898976 9379 ts_tablet_manager.cc:1428] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260809 00:59:41.900295 9381 raft_consensus.cc:348] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ede2103e44887993dd92a6bed0e80" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39129 } } peers { permanent_uuid: "162b73a90804441aad9a9c74be7856e4" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 40211 } } peers { permanent_uuid: "0c61c1aad8bb4187a31e0751826576ce" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 45379 } }
I20260809 00:59:41.900703 9381 raft_consensus.cc:374] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:41.900839 9381 raft_consensus.cc:729] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a44ede2103e44887993dd92a6bed0e80, State: Initialized, Role: FOLLOWER
I20260809 00:59:41.901335 9381 consensus_queue.cc:260] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ede2103e44887993dd92a6bed0e80" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39129 } } peers { permanent_uuid: "162b73a90804441aad9a9c74be7856e4" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 40211 } } peers { permanent_uuid: "0c61c1aad8bb4187a31e0751826576ce" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 45379 } }
I20260809 00:59:41.902356 9221 heartbeater.cc:503] Master 127.4.82.190:37517 was elected leader, sending a full tablet report...
I20260809 00:59:41.902854 9381 ts_tablet_manager.cc:1428] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260809 00:59:41.945081 9385 raft_consensus.cc:482] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:59:41.945334 9385 raft_consensus.cc:504] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ede2103e44887993dd92a6bed0e80" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39129 } } peers { permanent_uuid: "162b73a90804441aad9a9c74be7856e4" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 40211 } } peers { permanent_uuid: "0c61c1aad8bb4187a31e0751826576ce" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 45379 } }
I20260809 00:59:41.946944 9385 leader_election.cc:288] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a44ede2103e44887993dd92a6bed0e80 (127.4.82.130:39129), 0c61c1aad8bb4187a31e0751826576ce (127.4.82.131:45379)
I20260809 00:59:41.953840 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.947383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:34899 (local address 127.4.82.130:39129)
0809 00:59:41.947586 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:41.947594 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:41.948407 (+ 813us) server_negotiation.cc:407] Connection header received
0809 00:59:41.948560 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:41.948567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:41.948654 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:41.948746 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:41.949484 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.950234 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.952111 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.952888 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.953028 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.953117 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:41.953444 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 00:59:41.953571 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 00:59:41.953871 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.947456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:34203 (local address 127.4.82.131:45379)
0809 00:59:41.947677 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:41.947685 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:41.948573 (+ 888us) server_negotiation.cc:407] Connection header received
0809 00:59:41.948741 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:41.948747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:41.948821 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:41.948920 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:41.949573 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.950316 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.952163 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.952926 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.953108 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.953180 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:41.953507 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 00:59:41.953644 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 00:59:41.953966 9388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.947440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:39129 (local address 127.4.82.129:34899)
0809 00:59:41.948292 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:41.948307 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:41.948421 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:41.948776 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:41.948780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:41.948804 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:41.949312 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.949326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.950377 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.950384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.951948 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.951963 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.953094 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.953100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.953270 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.953310 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 00:59:41.953380 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:59:41.953506 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":21,"spinlock_wait_cycles":23296,"thread_start_us":466,"threads_started":1}
I20260809 00:59:41.954245 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.947487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:45379 (local address 127.4.82.129:34203)
0809 00:59:41.948425 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:41.948443 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:41.948591 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:41.948953 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:41.948956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:41.948984 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:41.949362 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.949371 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.950466 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.950470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.951983 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:41.951995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.953188 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.953193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:41.953325 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.953346 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 00:59:41.953458 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 00:59:41.953571 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":41,"thread_start_us":506,"threads_started":1}
I20260809 00:59:41.955238 9173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec01010b264d4ef2999fb0ab5f732901" candidate_uuid: "162b73a90804441aad9a9c74be7856e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a44ede2103e44887993dd92a6bed0e80" is_pre_election: true
I20260809 00:59:41.955226 9312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec01010b264d4ef2999fb0ab5f732901" candidate_uuid: "162b73a90804441aad9a9c74be7856e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c61c1aad8bb4187a31e0751826576ce" is_pre_election: true
I20260809 00:59:41.955814 9312 raft_consensus.cc:2454] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 162b73a90804441aad9a9c74be7856e4 in term 0.
I20260809 00:59:41.955794 9173 raft_consensus.cc:2454] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 162b73a90804441aad9a9c74be7856e4 in term 0.
I20260809 00:59:41.956501 8969 leader_election.cc:302] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0c61c1aad8bb4187a31e0751826576ce, 162b73a90804441aad9a9c74be7856e4, a44ede2103e44887993dd92a6bed0e80; no voters:
I20260809 00:59:41.956902 9385 raft_consensus.cc:2781] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:59:41.957062 9385 raft_consensus.cc:482] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:59:41.957177 9385 raft_consensus.cc:3037] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:41.959255 9385 raft_consensus.cc:504] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ede2103e44887993dd92a6bed0e80" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39129 } } peers { permanent_uuid: "162b73a90804441aad9a9c74be7856e4" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 40211 } } peers { permanent_uuid: "0c61c1aad8bb4187a31e0751826576ce" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 45379 } }
I20260809 00:59:41.959928 9385 leader_election.cc:288] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [CANDIDATE]: Term 1 election: Requested vote from peers a44ede2103e44887993dd92a6bed0e80 (127.4.82.130:39129), 0c61c1aad8bb4187a31e0751826576ce (127.4.82.131:45379)
I20260809 00:59:41.960289 9173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec01010b264d4ef2999fb0ab5f732901" candidate_uuid: "162b73a90804441aad9a9c74be7856e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a44ede2103e44887993dd92a6bed0e80"
I20260809 00:59:41.960489 9173 raft_consensus.cc:3037] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:41.960455 9312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec01010b264d4ef2999fb0ab5f732901" candidate_uuid: "162b73a90804441aad9a9c74be7856e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c61c1aad8bb4187a31e0751826576ce"
I20260809 00:59:41.960644 9312 raft_consensus.cc:3037] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:41.962809 9173 raft_consensus.cc:2454] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 162b73a90804441aad9a9c74be7856e4 in term 1.
I20260809 00:59:41.963011 9312 raft_consensus.cc:2454] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 162b73a90804441aad9a9c74be7856e4 in term 1.
I20260809 00:59:41.963171 8969 leader_election.cc:302] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 162b73a90804441aad9a9c74be7856e4, a44ede2103e44887993dd92a6bed0e80; no voters:
I20260809 00:59:41.963474 9385 raft_consensus.cc:2781] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:41.963897 9385 raft_consensus.cc:686] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [term 1 LEADER]: Becoming Leader. State: Replica: 162b73a90804441aad9a9c74be7856e4, State: Running, Role: LEADER
I20260809 00:59:41.964342 9385 consensus_queue.cc:237] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ede2103e44887993dd92a6bed0e80" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39129 } } peers { permanent_uuid: "162b73a90804441aad9a9c74be7856e4" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 40211 } } peers { permanent_uuid: "0c61c1aad8bb4187a31e0751826576ce" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 45379 } }
I20260809 00:59:41.968444 8886 catalog_manager.cc:5495] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 162b73a90804441aad9a9c74be7856e4 (127.4.82.129). New cstate: current_term: 1 leader_uuid: "162b73a90804441aad9a9c74be7856e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a44ede2103e44887993dd92a6bed0e80" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "162b73a90804441aad9a9c74be7856e4" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 40211 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c61c1aad8bb4187a31e0751826576ce" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 45379 } health_report { overall_health: UNKNOWN } } }
I20260809 00:59:41.984408 9376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:41.979234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56070 (local address 127.4.82.129:40211)
0809 00:59:41.979450 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:41.979457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:41.979476 (+ 19us) server_negotiation.cc:407] Connection header received
0809 00:59:41.979722 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:41.979726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:41.979811 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:41.979950 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:59:41.980324 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.981012 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.982319 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:41.982739 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:41.982886 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:41.983382 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:59:41.983823 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:59:41.983984 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:41.984102 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 00:59:41.984177 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 00:59:41.991886 9312 raft_consensus.cc:1261] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce [term 1 FOLLOWER]: Refusing update from remote peer 162b73a90804441aad9a9c74be7856e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:41.991891 9173 raft_consensus.cc:1261] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80 [term 1 FOLLOWER]: Refusing update from remote peer 162b73a90804441aad9a9c74be7856e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:41.992734 9385 consensus_queue.cc:1035] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a44ede2103e44887993dd92a6bed0e80" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:41.993010 9390 consensus_queue.cc:1035] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c61c1aad8bb4187a31e0751826576ce" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 45379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 00:59:41.993413 9086 tablet.cc:2369] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:59:41.998041 9362 tablet.cc:2369] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:59:42.004109 9393 mvcc.cc:205] Tried to move back new op lower bound from 7316427497429696512 to 7316427497329262592. Current Snapshot: MvccSnapshot[applied={T|T < 7316427497429696512}]
I20260809 00:59:42.005125 9396 mvcc.cc:205] Tried to move back new op lower bound from 7316427497429696512 to 7316427497329262592. Current Snapshot: MvccSnapshot[applied={T|T < 7316427497429696512}]
W20260809 00:59:42.005697 9222 tablet.cc:2369] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:59:42.017963 8886 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:42.018275 8886 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43164:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:42.021795 8886 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b244245ce7748e4bbfb4fdc84318872: Sending DeleteTablet for 3 replicas of tablet ec01010b264d4ef2999fb0ab5f732901
I20260809 00:59:42.022856 9153 tablet_service.cc:1514] Processing DeleteTablet for tablet ec01010b264d4ef2999fb0ab5f732901 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:42 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:53590
I20260809 00:59:42.022948 9016 tablet_service.cc:1514] Processing DeleteTablet for tablet ec01010b264d4ef2999fb0ab5f732901 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:42 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:56066
I20260809 00:59:42.023025 9290 tablet_service.cc:1514] Processing DeleteTablet for tablet ec01010b264d4ef2999fb0ab5f732901 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:42 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:53494
I20260809 00:59:42.028874 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 8953
I20260809 00:59:42.039469 9407 tablet_replica.cc:330] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce: stopping tablet replica
I20260809 00:59:42.040040 9407 raft_consensus.cc:2229] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:42.040695 9407 raft_consensus.cc:2258] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:42.043367 9407 ts_tablet_manager.cc:1905] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 00:59:42 dist-test-slave-07c2 krb5kdc[8842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 00:59:42.050531 9404 tablet_replica.cc:330] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80: stopping tablet replica
I20260809 00:59:42.051084 9404 raft_consensus.cc:2229] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:42.051390 9407 ts_tablet_manager.cc:1918] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:59:42.052018 9405 tablet_replica.cc:330] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4: stopping tablet replica
I20260809 00:59:42.052215 9404 raft_consensus.cc:2258] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:42.052551 9405 raft_consensus.cc:2229] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:59:42.053532 9405 raft_consensus.cc:2258] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:42.054394 9404 ts_tablet_manager.cc:1905] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:42.054644 9407 log.cc:1196] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/ec01010b264d4ef2999fb0ab5f732901
I20260809 00:59:42.055313 9407 ts_tablet_manager.cc:1939] T ec01010b264d4ef2999fb0ab5f732901 P 0c61c1aad8bb4187a31e0751826576ce: Deleting consensus metadata
I20260809 00:59:42.055934 9405 ts_tablet_manager.cc:1905] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:42.057302 8872 catalog_manager.cc:4841] TS 0c61c1aad8bb4187a31e0751826576ce (127.4.82.131:45379): tablet ec01010b264d4ef2999fb0ab5f732901 (table test-table [id=b9cafbd828fd4a169eecf2558be74104]) successfully deleted
I20260809 00:59:42.061123 9376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:42.030701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56082 (local address 127.4.82.129:40211)
0809 00:59:42.030918 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:42.030926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:42.031891 (+ 965us) server_negotiation.cc:407] Connection header received
0809 00:59:42.032583 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:42.032588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:42.032689 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:42.032779 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:42.037027 (+ 4248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:42.038652 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:42.042378 (+ 3726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:42.043078 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:42.043172 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:42.052496 (+ 9324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:42.052508 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:42.052533 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:42.052592 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:42.056750 (+ 4158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:42.057636 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:42.057644 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:42.057648 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:42.057833 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:42.058327 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:42.058332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:42.058335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:42.058778 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:42.059086 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:42.060444 (+ 1358us) server_negotiation.cc:299] Negotiation successful
0809 00:59:42.060696 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 00:59:42.061512 9404 ts_tablet_manager.cc:1918] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:59:42.061728 9404 log.cc:1196] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/ec01010b264d4ef2999fb0ab5f732901
I20260809 00:59:42.062059 9080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:42.062316 9404 ts_tablet_manager.cc:1939] T ec01010b264d4ef2999fb0ab5f732901 P a44ede2103e44887993dd92a6bed0e80: Deleting consensus metadata
I20260809 00:59:42.064924 8871 catalog_manager.cc:4841] TS a44ede2103e44887993dd92a6bed0e80 (127.4.82.130:39129): tablet ec01010b264d4ef2999fb0ab5f732901 (table test-table [id=b9cafbd828fd4a169eecf2558be74104]) successfully deleted
I20260809 00:59:42.158269 9405 ts_tablet_manager.cc:1918] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:59:42.158548 9405 log.cc:1196] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/ec01010b264d4ef2999fb0ab5f732901
I20260809 00:59:42.158960 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 8953
I20260809 00:59:42.159176 9405 ts_tablet_manager.cc:1939] T ec01010b264d4ef2999fb0ab5f732901 P 162b73a90804441aad9a9c74be7856e4: Deleting consensus metadata
W20260809 00:59:42.179576 8870 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.82.129:40211 (error 108)
I20260809 00:59:42.179848 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 9091
I20260809 00:59:42.181677 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:42.181194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:40211 (local address 127.0.0.1:56096)
0809 00:59:42.181385 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:42.181512 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:40211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 00:59:42.181941 8870 catalog_manager.cc:4584] TS 162b73a90804441aad9a9c74be7856e4 (127.4.82.129:40211): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ec01010b264d4ef2999fb0ab5f732901: Network error: Client connection negotiation failed: client connection to 127.4.82.129:40211: connect: Connection refused (error 111)
Aug 09 00:59:42 dist-test-slave-07c2 krb5kdc[8842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 00:59:42.194940 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:42.180774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53604 (local address 127.4.82.130:39129)
0809 00:59:42.180950 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:42.180956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:42.180975 (+ 19us) server_negotiation.cc:407] Connection header received
0809 00:59:42.181169 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:42.181174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:42.181310 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:42.181417 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:42.183011 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:42.183873 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:42.185067 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:42.185436 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:42.185522 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:42.189386 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:42.189398 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:42.189411 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:42.189445 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:42.192413 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:42.193213 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:42.193221 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:42.193223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:42.193340 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:42.193669 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:42.193672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:42.193674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:42.194014 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:42.194222 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:42.194426 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 00:59:42.194582 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 00:59:42.195653 9216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:42.235761 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:42.235252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:40211 (local address 127.0.0.1:56104)
0809 00:59:42.235495 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:42.235595 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:40211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260809 00:59:42.236789 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:42.236418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:40211 (local address 127.0.0.1:56106)
0809 00:59:42.236566 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:42.236668 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:40211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 00:59:42.283388 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 9091
I20260809 00:59:42.297245 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:42.296626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:40211 (local address 127.0.0.1:56122)
0809 00:59:42.296916 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:42.297032 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:40211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 00:59:42.298441 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:42.297988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:40211 (local address 127.0.0.1:56128)
0809 00:59:42.298174 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:42.298276 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:40211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 00:59:42.302937 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 9226
Aug 09 00:59:42 dist-test-slave-07c2 krb5kdc[8842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 00:59:42.317793 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:42.303693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53496 (local address 127.4.82.131:45379)
0809 00:59:42.303890 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:42.303897 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:42.303919 (+ 22us) server_negotiation.cc:407] Connection header received
0809 00:59:42.303983 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:42.303989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:42.304068 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:42.304156 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:42.305418 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:42.306111 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:42.307253 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:42.307771 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:42.307862 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:42.311985 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:42.311998 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:42.312014 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:42.312049 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:42.315344 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:42.316166 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:42.316174 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:42.316176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:42.316304 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:42.316620 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:42.316624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:42.316625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:42.316985 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:42.317188 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:42.317407 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 00:59:42.317558 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 00:59:42.318451 9356 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:42.379237 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:42.378658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:40211 (local address 127.0.0.1:56130)
0809 00:59:42.378987 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:42.379082 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:40211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260809 00:59:42.380307 9375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:42.379954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:40211 (local address 127.0.0.1:56136)
0809 00:59:42.380137 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:42.380204 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:40211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260809 00:59:42.409299 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 9226
I20260809 00:59:42.428314 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 8855
I20260809 00:59:42.429430 8920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:42.539526 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 8855
2026-08-09T00:59:42Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4104 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T00:59:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:42Z Disabled control of system clock
I20260809 00:59:42.611096 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:44627
--webserver_interface=127.4.82.190
--webserver_port=0
--builtin_ntp_servers=127.4.82.148:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:44627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 00:59:42.928602 9417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:42.928865 9417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:42.928946 9417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:42.935703 9417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:42.935792 9417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:42.935827 9417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:42.935886 9417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:42.935928 9417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:42.944525 9417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:44627
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:44627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:42.945550 9417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:42.947459 9417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:42.958630 9425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:42.958690 9422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:42.958660 9423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:42.959774 9417 server_base.cc:1034] running on GCE node
I20260809 00:59:42.960445 9417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:42.961658 9417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:42.962862 9417 hybrid_clock.cc:648] HybridClock initialized: now 1786237182962810 us; error 49 us; skew 500 ppm
I20260809 00:59:42.963240 9417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 00:59:42.965436 9417 webserver.cc:460] Webserver started at http://127.4.82.190:40901/ using document root <none> and password file <none>
I20260809 00:59:42.966068 9417 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:42.966180 9417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:42.966495 9417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:42.969249 9417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "1e000f893edf4cd080929bebfb40c245"
format_stamp: "Formatted at 2026-08-09 00:59:42 on dist-test-slave-07c2"
server_key: "e0fbdb564d177aa35ddb16a06d9b7386"
server_key_iv: "7a129c90b028db186159130295cf71f8"
server_key_version: "encryptionkey@0"
I20260809 00:59:42.970022 9417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "1e000f893edf4cd080929bebfb40c245"
format_stamp: "Formatted at 2026-08-09 00:59:42 on dist-test-slave-07c2"
server_key: "e0fbdb564d177aa35ddb16a06d9b7386"
server_key_iv: "7a129c90b028db186159130295cf71f8"
server_key_version: "encryptionkey@0"
I20260809 00:59:42.974691 9417 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 00:59:42.977891 9431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:42.979096 9417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 00:59:42.979271 9417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "1e000f893edf4cd080929bebfb40c245"
format_stamp: "Formatted at 2026-08-09 00:59:42 on dist-test-slave-07c2"
server_key: "e0fbdb564d177aa35ddb16a06d9b7386"
server_key_iv: "7a129c90b028db186159130295cf71f8"
server_key_version: "encryptionkey@0"
I20260809 00:59:42.979538 9417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:42.984899 9417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:42.985586 9417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:42.985881 9417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:43.004386 9417 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:44627
I20260809 00:59:43.004428 9486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:44627 every 8 connection(s)
I20260809 00:59:43.005622 9417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:43.007783 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 9417
I20260809 00:59:43.007989 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:43.008857 4426 external_mini_cluster.cc:1428] Setting key cad1f17c673d508977f13c8a47b159ac
I20260809 00:59:43.009894 9487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:43.017408 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:43.010869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59966 (local address 127.4.82.190:44627)
0809 00:59:43.011836 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:43.011846 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:43.011885 (+ 39us) server_negotiation.cc:407] Connection header received
0809 00:59:43.014131 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:43.014151 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:43.014518 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:43.014832 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:43.015949 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:43.015954 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:43.015970 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:43.015986 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:43.016043 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:43.016057 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:43.016232 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:43.016328 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 00:59:43.016568 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":404,"threads_started":1}
I20260809 00:59:43.022014 9487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245: Bootstrap starting.
I20260809 00:59:43.024768 9487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:43.025761 9487 log.cc:824] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:43.028220 9487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245: No bootstrap required, opened a new log
I20260809 00:59:43.034870 9487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e000f893edf4cd080929bebfb40c245" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44627 } }
I20260809 00:59:43.035211 9487 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:43.035351 9487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e000f893edf4cd080929bebfb40c245, State: Initialized, Role: FOLLOWER
I20260809 00:59:43.035866 9487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e000f893edf4cd080929bebfb40c245" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44627 } }
I20260809 00:59:43.036106 9487 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:43.036257 9487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:43.036424 9487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:43.038412 9487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e000f893edf4cd080929bebfb40c245" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44627 } }
I20260809 00:59:43.038993 9487 leader_election.cc:302] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e000f893edf4cd080929bebfb40c245; no voters:
I20260809 00:59:43.039642 9487 leader_election.cc:288] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:43.039749 9492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:43.040498 9492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [term 1 LEADER]: Becoming Leader. State: Replica: 1e000f893edf4cd080929bebfb40c245, State: Running, Role: LEADER
I20260809 00:59:43.040983 9492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e000f893edf4cd080929bebfb40c245" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44627 } }
I20260809 00:59:43.041625 9487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:43.043756 9494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e000f893edf4cd080929bebfb40c245. Latest consensus state: current_term: 1 leader_uuid: "1e000f893edf4cd080929bebfb40c245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e000f893edf4cd080929bebfb40c245" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44627 } } }
I20260809 00:59:43.044210 9494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:43.044003 9493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e000f893edf4cd080929bebfb40c245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e000f893edf4cd080929bebfb40c245" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44627 } } }
I20260809 00:59:43.044327 9493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:43.046073 9501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:43.050781 9501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:43.059532 9501 catalog_manager.cc:1347] Generated new cluster ID: 61563279cf0f4fcdb97d7e7c1f382c97
I20260809 00:59:43.059633 9501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:43.083065 9501 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:43.083508 9501 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:43.089968 9501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245: Generated new TSK 0
I20260809 00:59:43.090842 9501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:59:43.101778 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44627
--builtin_ntp_servers=127.4.82.148:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 00:59:43.423509 9513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:43.423808 9513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:43.423964 9513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:43.430861 9513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:43.430981 9513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:43.431154 9513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 00:59:43.439898 9513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:43.441083 9513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:43.442947 9513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:43.455687 9518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:43.455600 9519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:43.455654 9521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:43.456489 9513 server_base.cc:1034] running on GCE node
I20260809 00:59:43.457154 9513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:43.458086 9513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:43.459352 9513 hybrid_clock.cc:648] HybridClock initialized: now 1786237183459249 us; error 49 us; skew 500 ppm
I20260809 00:59:43.459685 9513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 00:59:43.461766 9513 webserver.cc:460] Webserver started at http://127.4.82.129:32993/ using document root <none> and password file <none>
I20260809 00:59:43.462389 9513 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:43.462523 9513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:43.462919 9513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:43.465718 9513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "4b77606586d24bdbaaecbc885de5a025"
format_stamp: "Formatted at 2026-08-09 00:59:43 on dist-test-slave-07c2"
server_key: "4dc528c88b87a75d71c1d38069a9e438"
server_key_iv: "aa55da50c3e9420d5fc51ace19ea849d"
server_key_version: "encryptionkey@0"
I20260809 00:59:43.466487 9513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "4b77606586d24bdbaaecbc885de5a025"
format_stamp: "Formatted at 2026-08-09 00:59:43 on dist-test-slave-07c2"
server_key: "4dc528c88b87a75d71c1d38069a9e438"
server_key_iv: "aa55da50c3e9420d5fc51ace19ea849d"
server_key_version: "encryptionkey@0"
I20260809 00:59:43.471204 9513 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 00:59:43.474360 9527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:43.475651 9513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 00:59:43.475833 9513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "4b77606586d24bdbaaecbc885de5a025"
format_stamp: "Formatted at 2026-08-09 00:59:43 on dist-test-slave-07c2"
server_key: "4dc528c88b87a75d71c1d38069a9e438"
server_key_iv: "aa55da50c3e9420d5fc51ace19ea849d"
server_key_version: "encryptionkey@0"
I20260809 00:59:43.476063 9513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:43.481329 9513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:43.482079 9513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:43.482339 9513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:43.483358 9513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:43.484771 9513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:43.484891 9513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:43.485019 9513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:43.485118 9513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:43.517392 9513 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:44223
I20260809 00:59:43.517474 9651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:44223 every 8 connection(s)
I20260809 00:59:43.518667 9513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 00:59:43.523516 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 9513
I20260809 00:59:43.523696 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 00:59:43.524309 4426 external_mini_cluster.cc:1428] Setting key 67ef02e2a1ad8d775bebf9aa4383ce12
I20260809 00:59:43.526706 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:43.521162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:41251 (local address 127.4.82.190:44627)
0809 00:59:43.521598 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:43.521602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:43.522640 (+ 1038us) server_negotiation.cc:407] Connection header received
0809 00:59:43.524500 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:43.524506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:43.524578 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:43.524698 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:43.525972 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:43.525982 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:43.525985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:43.526000 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:43.526040 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:43.526052 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:43.526173 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:43.526360 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 00:59:43.526427 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":231,"threads_started":1}
I20260809 00:59:43.527041 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44627
--builtin_ntp_servers=127.4.82.148:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 00:59:43.527853 9655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:43.521631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44627 (local address 127.4.82.129:41251)
0809 00:59:43.522487 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:43.522531 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:43.524196 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:43.524932 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:43.524938 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:43.525017 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:43.525680 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:59:43.525682 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:43.525725 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:59:43.525810 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:43.526147 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:43.526157 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:43.526165 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 00:59:43.526439 (+ 274us) client_negotiation.cc:240] Negotiation successful
0809 00:59:43.526663 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":273,"threads_started":1}
I20260809 00:59:43.529270 9652 heartbeater.cc:347] Connected to a master server at 127.4.82.190:44627
I20260809 00:59:43.529551 9652 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:43.530076 9652 heartbeater.cc:511] Master 127.4.82.190:44627 requested a full tablet report, sending...
I20260809 00:59:43.532307 9449 ts_manager.cc:194] Registered new tserver with Master: 4b77606586d24bdbaaecbc885de5a025 (127.4.82.129:44223)
W20260809 00:59:43.849792 9656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:43.850065 9656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:43.850214 9656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:43.857353 9656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:43.857443 9656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:43.857592 9656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 00:59:43.866475 9656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:43.867729 9656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:43.869455 9656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:43.882040 9662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:43.881994 9665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:43.882092 9663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:43.882882 9656 server_base.cc:1034] running on GCE node
I20260809 00:59:43.883554 9656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:43.884500 9656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:43.885725 9656 hybrid_clock.cc:648] HybridClock initialized: now 1786237183885684 us; error 35 us; skew 500 ppm
I20260809 00:59:43.886034 9656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 00:59:43.888387 9656 webserver.cc:460] Webserver started at http://127.4.82.130:33189/ using document root <none> and password file <none>
I20260809 00:59:43.889055 9656 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:43.889173 9656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:43.889493 9656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:43.892320 9656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "a6debb1192854880b0b340a3a5f5ab1e"
format_stamp: "Formatted at 2026-08-09 00:59:43 on dist-test-slave-07c2"
server_key: "be5d5d68d729e0c045bf4b6429f0a0a6"
server_key_iv: "f8708dcbebc9eec8b578ed1580d56d6a"
server_key_version: "encryptionkey@0"
I20260809 00:59:43.893092 9656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "a6debb1192854880b0b340a3a5f5ab1e"
format_stamp: "Formatted at 2026-08-09 00:59:43 on dist-test-slave-07c2"
server_key: "be5d5d68d729e0c045bf4b6429f0a0a6"
server_key_iv: "f8708dcbebc9eec8b578ed1580d56d6a"
server_key_version: "encryptionkey@0"
I20260809 00:59:43.898077 9656 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 00:59:43.901377 9671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:43.902626 9656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 00:59:43.902814 9656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "a6debb1192854880b0b340a3a5f5ab1e"
format_stamp: "Formatted at 2026-08-09 00:59:43 on dist-test-slave-07c2"
server_key: "be5d5d68d729e0c045bf4b6429f0a0a6"
server_key_iv: "f8708dcbebc9eec8b578ed1580d56d6a"
server_key_version: "encryptionkey@0"
I20260809 00:59:43.903046 9656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:43.908442 9656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:43.909205 9656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:43.909479 9656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:43.910344 9656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:43.911748 9656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:43.911870 9656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:43.912005 9656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:43.912104 9656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:43.945329 9656 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:39901
I20260809 00:59:43.945382 9790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:39901 every 8 connection(s)
I20260809 00:59:43.946821 9656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 00:59:43.949002 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 9656
I20260809 00:59:43.949174 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 00:59:43.949792 4426 external_mini_cluster.cc:1428] Setting key 94777742fd03caea6f95614e03da8a8c
I20260809 00:59:43.952932 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44627
--builtin_ntp_servers=127.4.82.148:45829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 00:59:43.955741 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:43.949451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:35437 (local address 127.4.82.190:44627)
0809 00:59:43.949632 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:43.949636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:43.951454 (+ 1818us) server_negotiation.cc:407] Connection header received
0809 00:59:43.953343 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:43.953349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:43.953416 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:43.953520 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:43.954912 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:43.954920 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:43.954927 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:43.954950 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:43.954994 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:43.955012 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:43.955146 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:43.955331 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 00:59:43.955398 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 00:59:43.956517 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:43.950086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44627 (local address 127.4.82.130:35437)
0809 00:59:43.951263 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:43.951334 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:43.953042 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:43.953743 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:43.953750 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:43.953834 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:43.954554 (+ 720us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:59:43.954558 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:43.954607 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:59:43.954719 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:43.955117 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:43.955124 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:43.955130 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 00:59:43.955389 (+ 259us) client_negotiation.cc:240] Negotiation successful
0809 00:59:43.955564 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":509,"threads_started":1}
I20260809 00:59:43.957813 9791 heartbeater.cc:347] Connected to a master server at 127.4.82.190:44627
I20260809 00:59:43.958138 9791 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:43.958793 9791 heartbeater.cc:511] Master 127.4.82.190:44627 requested a full tablet report, sending...
I20260809 00:59:43.960278 9449 ts_manager.cc:194] Registered new tserver with Master: a6debb1192854880b0b340a3a5f5ab1e (127.4.82.130:39901)
W20260809 00:59:44.274727 9794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:44.274993 9794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:44.275157 9794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:44.282259 9794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:44.282346 9794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:44.282545 9794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 00:59:44.291483 9794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:45829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44627
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:44.292698 9794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:44.294486 9794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:44.307156 9799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:44.307248 9800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:44.307272 9802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:44.308199 9794 server_base.cc:1034] running on GCE node
I20260809 00:59:44.308838 9794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:44.309860 9794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:44.311084 9794 hybrid_clock.cc:648] HybridClock initialized: now 1786237184311053 us; error 49 us; skew 500 ppm
I20260809 00:59:44.311460 9794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 00:59:44.313620 9794 webserver.cc:460] Webserver started at http://127.4.82.131:42535/ using document root <none> and password file <none>
I20260809 00:59:44.314301 9794 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:44.314445 9794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:44.314787 9794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:44.317582 9794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "e16b8c1397684a7db520a7bb7ec22767"
format_stamp: "Formatted at 2026-08-09 00:59:44 on dist-test-slave-07c2"
server_key: "83931baf353b549d8a0a6bae6c54b070"
server_key_iv: "c078cc9a33829ab836f2af357c13aa05"
server_key_version: "encryptionkey@0"
I20260809 00:59:44.318368 9794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "e16b8c1397684a7db520a7bb7ec22767"
format_stamp: "Formatted at 2026-08-09 00:59:44 on dist-test-slave-07c2"
server_key: "83931baf353b549d8a0a6bae6c54b070"
server_key_iv: "c078cc9a33829ab836f2af357c13aa05"
server_key_version: "encryptionkey@0"
I20260809 00:59:44.323374 9794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 00:59:44.326563 9808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:44.327988 9794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 00:59:44.328162 9794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "e16b8c1397684a7db520a7bb7ec22767"
format_stamp: "Formatted at 2026-08-09 00:59:44 on dist-test-slave-07c2"
server_key: "83931baf353b549d8a0a6bae6c54b070"
server_key_iv: "c078cc9a33829ab836f2af357c13aa05"
server_key_version: "encryptionkey@0"
I20260809 00:59:44.328393 9794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:44.333947 9794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:44.334692 9794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:44.334964 9794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:44.335938 9794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:44.337244 9794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:44.337364 9794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:44.337498 9794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:44.337589 9794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:44.371860 9794 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:42845
I20260809 00:59:44.371917 9928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:42845 every 8 connection(s)
I20260809 00:59:44.373234 9794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 00:59:44.375978 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 9794
I20260809 00:59:44.376150 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 00:59:44.376645 4426 external_mini_cluster.cc:1428] Setting key a9b931851f117eb7a0204184467e9a5a
I20260809 00:59:44.381772 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.375929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:47383 (local address 127.4.82.190:44627)
0809 00:59:44.376175 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:44.376183 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:44.377800 (+ 1617us) server_negotiation.cc:407] Connection header received
0809 00:59:44.379789 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:44.379796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:44.379866 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:44.379969 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:44.381201 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:44.381205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:44.381208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:44.381223 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:44.381253 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:44.381265 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:44.381363 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:44.381493 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 00:59:44.381540 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 00:59:44.382622 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.376533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44627 (local address 127.4.82.131:47383)
0809 00:59:44.377631 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:44.377674 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:44.379465 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:44.380177 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:44.380184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:44.380305 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:44.380935 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:59:44.380937 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:44.380972 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:59:44.381054 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:44.381333 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:44.381338 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:44.381343 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:59:44.381535 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 00:59:44.381721 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":409,"threads_started":1}
I20260809 00:59:44.384234 9929 heartbeater.cc:347] Connected to a master server at 127.4.82.190:44627
I20260809 00:59:44.384541 9929 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:44.385116 9929 heartbeater.cc:511] Master 127.4.82.190:44627 requested a full tablet report, sending...
I20260809 00:59:44.386443 9449 ts_manager.cc:194] Registered new tserver with Master: e16b8c1397684a7db520a7bb7ec22767 (127.4.82.131:42845)
I20260809 00:59:44.391911 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:59:44.398706 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.396411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59970 (local address 127.4.82.190:44627)
0809 00:59:44.396564 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:44.396568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:44.397132 (+ 564us) server_negotiation.cc:407] Connection header received
0809 00:59:44.397245 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:44.397248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:44.397300 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:44.397378 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:44.398212 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:44.398216 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:44.398219 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:44.398234 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:44.398263 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:44.398275 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:44.398370 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:44.398451 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 00:59:44.398496 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 00:59:44.403616 9449 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:59:44.410705 9449 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:59:44.437855 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.422511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41892 (local address 127.4.82.129:44223)
0809 00:59:44.434153 (+ 11642us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:44.434164 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:44.434198 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:59:44.434278 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:44.434290 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:44.434547 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:44.435447 (+ 900us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:44.436944 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:44.436951 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:44.436961 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:44.436980 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:44.437019 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:44.437033 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:44.437184 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:44.437376 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 00:59:44.437477 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11438,"thread_start_us":11298,"threads_started":1}
I20260809 00:59:44.437983 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.422402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:44223 (local address 127.0.0.1:41892)
0809 00:59:44.433359 (+ 10957us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:44.433415 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:44.433566 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:44.434785 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:44.434794 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:44.434888 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:44.435814 (+ 926us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:59:44.435818 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:44.435853 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:59:44.435950 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:44.437147 (+ 1197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:44.437169 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:44.437176 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 00:59:44.437392 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 00:59:44.437477 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10686,"thread_start_us":14313,"threads_started":1}
I20260809 00:59:44.439740 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.423445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42098 (local address 127.4.82.130:39901)
0809 00:59:44.434492 (+ 11047us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:44.434504 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:44.435865 (+ 1361us) server_negotiation.cc:407] Connection header received
0809 00:59:44.436148 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:44.436160 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:44.436368 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:44.436476 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:44.438811 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:44.438820 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:44.438832 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:44.438861 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:44.438920 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:44.438942 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:44.439133 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:44.439238 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 00:59:44.439362 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10841,"thread_start_us":10703,"threads_started":1}
I20260809 00:59:44.439980 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.422604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:39901 (local address 127.0.0.1:42098)
0809 00:59:44.435718 (+ 13114us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:44.435744 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:44.435955 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:44.437615 (+ 1660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:44.437620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:44.437644 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:44.438528 (+ 884us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:59:44.438531 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:44.438544 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:59:44.438612 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:44.439052 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:44.439056 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:44.439061 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:59:44.439188 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 00:59:44.439243 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12925,"thread_start_us":12789,"threads_started":1}
I20260809 00:59:44.440708 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.423602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40038 (local address 127.4.82.131:42845)
0809 00:59:44.435166 (+ 11564us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:44.435187 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:44.436234 (+ 1047us) server_negotiation.cc:407] Connection header received
0809 00:59:44.436314 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:44.436323 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:44.436489 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:44.436644 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:44.438995 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:44.439004 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:44.439021 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:44.439052 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:44.439119 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:44.439144 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:44.439336 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:44.440182 (+ 846us) server_negotiation.cc:299] Negotiation successful
0809 00:59:44.440284 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11348,"thread_start_us":11120,"threads_started":1}
I20260809 00:59:44.440057 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.422934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:42845 (local address 127.0.0.1:40038)
0809 00:59:44.436011 (+ 13077us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:44.436029 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:44.436127 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:44.437634 (+ 1507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:44.437637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:44.437652 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:44.438779 (+ 1127us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:59:44.438786 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:44.438799 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:59:44.438828 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:44.439399 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:44.439405 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:44.439410 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:59:44.439550 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 00:59:44.439598 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12953,"thread_start_us":8708,"threads_started":1}
I20260809 00:59:44.442314 9580 tablet_service.cc:1467] Processing CreateTablet for tablet 89d1116986854f95a4801156878cbb21 (DEFAULT_TABLE table=test-table [id=c62a663b5cd5449297d1a2c1921f6746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:44.443692 9580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89d1116986854f95a4801156878cbb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:44.443897 9721 tablet_service.cc:1467] Processing CreateTablet for tablet 89d1116986854f95a4801156878cbb21 (DEFAULT_TABLE table=test-table [id=c62a663b5cd5449297d1a2c1921f6746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:44.444563 9859 tablet_service.cc:1467] Processing CreateTablet for tablet 89d1116986854f95a4801156878cbb21 (DEFAULT_TABLE table=test-table [id=c62a663b5cd5449297d1a2c1921f6746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:44.445281 9721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89d1116986854f95a4801156878cbb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:44.445917 9859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89d1116986854f95a4801156878cbb21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:44.456997 9947 tablet_bootstrap.cc:492] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025: Bootstrap starting.
I20260809 00:59:44.459976 9948 tablet_bootstrap.cc:492] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e: Bootstrap starting.
I20260809 00:59:44.460507 9947 tablet_bootstrap.cc:654] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:44.461936 9947 log.cc:824] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:44.461989 9949 tablet_bootstrap.cc:492] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767: Bootstrap starting.
I20260809 00:59:44.463583 9948 tablet_bootstrap.cc:654] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:44.465122 9949 tablet_bootstrap.cc:654] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:44.465116 9948 log.cc:824] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:44.466313 9949 log.cc:824] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:44.468020 9947 tablet_bootstrap.cc:492] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025: No bootstrap required, opened a new log
I20260809 00:59:44.468020 9948 tablet_bootstrap.cc:492] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e: No bootstrap required, opened a new log
I20260809 00:59:44.468299 9948 ts_tablet_manager.cc:1397] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260809 00:59:44.468299 9947 ts_tablet_manager.cc:1397] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260809 00:59:44.469028 9949 tablet_bootstrap.cc:492] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767: No bootstrap required, opened a new log
I20260809 00:59:44.469298 9949 ts_tablet_manager.cc:1397] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 00:59:44.477265 9949 raft_consensus.cc:348] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b77606586d24bdbaaecbc885de5a025" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44223 } } peers { permanent_uuid: "a6debb1192854880b0b340a3a5f5ab1e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39901 } } peers { permanent_uuid: "e16b8c1397684a7db520a7bb7ec22767" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42845 } }
I20260809 00:59:44.477686 9949 raft_consensus.cc:374] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:44.477797 9949 raft_consensus.cc:729] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e16b8c1397684a7db520a7bb7ec22767, State: Initialized, Role: FOLLOWER
I20260809 00:59:44.478428 9949 consensus_queue.cc:260] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b77606586d24bdbaaecbc885de5a025" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44223 } } peers { permanent_uuid: "a6debb1192854880b0b340a3a5f5ab1e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39901 } } peers { permanent_uuid: "e16b8c1397684a7db520a7bb7ec22767" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42845 } }
I20260809 00:59:44.478953 9947 raft_consensus.cc:348] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b77606586d24bdbaaecbc885de5a025" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44223 } } peers { permanent_uuid: "a6debb1192854880b0b340a3a5f5ab1e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39901 } } peers { permanent_uuid: "e16b8c1397684a7db520a7bb7ec22767" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42845 } }
I20260809 00:59:44.479058 9948 raft_consensus.cc:348] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b77606586d24bdbaaecbc885de5a025" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44223 } } peers { permanent_uuid: "a6debb1192854880b0b340a3a5f5ab1e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39901 } } peers { permanent_uuid: "e16b8c1397684a7db520a7bb7ec22767" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42845 } }
I20260809 00:59:44.479445 9947 raft_consensus.cc:374] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:44.479602 9947 raft_consensus.cc:729] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b77606586d24bdbaaecbc885de5a025, State: Initialized, Role: FOLLOWER
I20260809 00:59:44.479713 9929 heartbeater.cc:503] Master 127.4.82.190:44627 was elected leader, sending a full tablet report...
I20260809 00:59:44.480234 9947 consensus_queue.cc:260] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b77606586d24bdbaaecbc885de5a025" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44223 } } peers { permanent_uuid: "a6debb1192854880b0b340a3a5f5ab1e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39901 } } peers { permanent_uuid: "e16b8c1397684a7db520a7bb7ec22767" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42845 } }
I20260809 00:59:44.480453 9949 ts_tablet_manager.cc:1428] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767: Time spent starting tablet: real 0.011s user 0.006s sys 0.006s
I20260809 00:59:44.479482 9948 raft_consensus.cc:374] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:44.481096 9948 raft_consensus.cc:729] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6debb1192854880b0b340a3a5f5ab1e, State: Initialized, Role: FOLLOWER
I20260809 00:59:44.481352 9652 heartbeater.cc:503] Master 127.4.82.190:44627 was elected leader, sending a full tablet report...
I20260809 00:59:44.481988 9947 ts_tablet_manager.cc:1428] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 00:59:44.481817 9948 consensus_queue.cc:260] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b77606586d24bdbaaecbc885de5a025" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44223 } } peers { permanent_uuid: "a6debb1192854880b0b340a3a5f5ab1e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39901 } } peers { permanent_uuid: "e16b8c1397684a7db520a7bb7ec22767" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42845 } }
I20260809 00:59:44.489506 9791 heartbeater.cc:503] Master 127.4.82.190:44627 was elected leader, sending a full tablet report...
I20260809 00:59:44.490129 9948 ts_tablet_manager.cc:1428] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e: Time spent starting tablet: real 0.022s user 0.006s sys 0.013s
W20260809 00:59:44.520890 9653 tablet.cc:2369] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:59:44.625032 9930 tablet.cc:2369] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:59:44.687433 9955 raft_consensus.cc:482] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:59:44.687779 9955 raft_consensus.cc:504] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b77606586d24bdbaaecbc885de5a025" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44223 } } peers { permanent_uuid: "a6debb1192854880b0b340a3a5f5ab1e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39901 } } peers { permanent_uuid: "e16b8c1397684a7db520a7bb7ec22767" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42845 } }
I20260809 00:59:44.689415 9955 leader_election.cc:288] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b77606586d24bdbaaecbc885de5a025 (127.4.82.129:44223), e16b8c1397684a7db520a7bb7ec22767 (127.4.82.131:42845)
I20260809 00:59:44.693171 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.689840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:55295 (local address 127.4.82.129:44223)
0809 00:59:44.690037 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:44.690043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:44.691157 (+ 1114us) server_negotiation.cc:407] Connection header received
0809 00:59:44.691353 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:44.691361 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:44.691432 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:44.691517 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:44.692531 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:44.692537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:44.692542 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:44.692567 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:44.692598 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:44.692610 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:44.692765 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:44.692811 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 00:59:44.692878 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 00:59:44.693333 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.689847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:48121 (local address 127.4.82.131:42845)
0809 00:59:44.690037 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:44.690043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:44.691248 (+ 1205us) server_negotiation.cc:407] Connection header received
0809 00:59:44.691434 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:44.691438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:44.691506 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:44.691605 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:44.692724 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:44.692729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:44.692733 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:44.692753 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:44.692790 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:44.692805 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:44.692926 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:44.693022 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 00:59:44.693078 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 00:59:44.693650 9956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.689939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:42845 (local address 127.4.82.130:48121)
0809 00:59:44.691090 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:44.691125 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:44.691288 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:44.691649 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:44.691654 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:44.691680 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:44.692524 (+ 844us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:59:44.692527 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:44.692538 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:59:44.692571 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:44.692917 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:44.692922 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:44.692926 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:59:44.693062 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 00:59:44.693115 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"mutex_wait_us":23,"spinlock_wait_cycles":20736,"thread_start_us":597,"threads_started":1}
I20260809 00:59:44.693800 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.689939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:44223 (local address 127.4.82.130:55295)
0809 00:59:44.691041 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:44.691057 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:44.691171 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:44.691577 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:44.691585 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:44.691609 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:44.692325 (+ 716us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:59:44.692327 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:44.692341 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:59:44.692377 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:44.692728 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:44.692731 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:44.692733 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:59:44.692830 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 00:59:44.692947 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"thread_start_us":669,"threads_started":1}
I20260809 00:59:44.694700 9880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d1116986854f95a4801156878cbb21" candidate_uuid: "a6debb1192854880b0b340a3a5f5ab1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e16b8c1397684a7db520a7bb7ec22767" is_pre_election: true
I20260809 00:59:44.694756 9601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d1116986854f95a4801156878cbb21" candidate_uuid: "a6debb1192854880b0b340a3a5f5ab1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b77606586d24bdbaaecbc885de5a025" is_pre_election: true
I20260809 00:59:44.695194 9880 raft_consensus.cc:2454] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6debb1192854880b0b340a3a5f5ab1e in term 0.
I20260809 00:59:44.695320 9601 raft_consensus.cc:2454] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6debb1192854880b0b340a3a5f5ab1e in term 0.
I20260809 00:59:44.695883 9672 leader_election.cc:302] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6debb1192854880b0b340a3a5f5ab1e, e16b8c1397684a7db520a7bb7ec22767; no voters:
I20260809 00:59:44.696239 9955 raft_consensus.cc:2781] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:59:44.696384 9955 raft_consensus.cc:482] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:59:44.696481 9955 raft_consensus.cc:3037] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [term 0 FOLLOWER]: Advancing to term 1
W20260809 00:59:44.698836 9792 tablet.cc:2369] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:59:44.699230 9955 raft_consensus.cc:504] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b77606586d24bdbaaecbc885de5a025" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44223 } } peers { permanent_uuid: "a6debb1192854880b0b340a3a5f5ab1e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39901 } } peers { permanent_uuid: "e16b8c1397684a7db520a7bb7ec22767" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42845 } }
I20260809 00:59:44.699939 9955 leader_election.cc:288] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [CANDIDATE]: Term 1 election: Requested vote from peers 4b77606586d24bdbaaecbc885de5a025 (127.4.82.129:44223), e16b8c1397684a7db520a7bb7ec22767 (127.4.82.131:42845)
I20260809 00:59:44.700548 9601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d1116986854f95a4801156878cbb21" candidate_uuid: "a6debb1192854880b0b340a3a5f5ab1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b77606586d24bdbaaecbc885de5a025"
I20260809 00:59:44.700582 9880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89d1116986854f95a4801156878cbb21" candidate_uuid: "a6debb1192854880b0b340a3a5f5ab1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e16b8c1397684a7db520a7bb7ec22767"
I20260809 00:59:44.700767 9601 raft_consensus.cc:3037] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:44.700790 9880 raft_consensus.cc:3037] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:44.704047 9601 raft_consensus.cc:2454] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6debb1192854880b0b340a3a5f5ab1e in term 1.
I20260809 00:59:44.704047 9880 raft_consensus.cc:2454] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6debb1192854880b0b340a3a5f5ab1e in term 1.
I20260809 00:59:44.704445 9675 leader_election.cc:302] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b77606586d24bdbaaecbc885de5a025, a6debb1192854880b0b340a3a5f5ab1e; no voters:
I20260809 00:59:44.704746 9955 raft_consensus.cc:2781] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:44.705294 9955 raft_consensus.cc:686] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [term 1 LEADER]: Becoming Leader. State: Replica: a6debb1192854880b0b340a3a5f5ab1e, State: Running, Role: LEADER
I20260809 00:59:44.705826 9955 consensus_queue.cc:237] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b77606586d24bdbaaecbc885de5a025" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44223 } } peers { permanent_uuid: "a6debb1192854880b0b340a3a5f5ab1e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39901 } } peers { permanent_uuid: "e16b8c1397684a7db520a7bb7ec22767" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42845 } }
I20260809 00:59:44.710281 9449 catalog_manager.cc:5495] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e reported cstate change: term changed from 0 to 1, leader changed from <none> to a6debb1192854880b0b340a3a5f5ab1e (127.4.82.130). New cstate: current_term: 1 leader_uuid: "a6debb1192854880b0b340a3a5f5ab1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b77606586d24bdbaaecbc885de5a025" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6debb1192854880b0b340a3a5f5ab1e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 39901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e16b8c1397684a7db520a7bb7ec22767" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42845 } health_report { overall_health: UNKNOWN } } }
I20260809 00:59:44.736524 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.733889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42100 (local address 127.4.82.130:39901)
0809 00:59:44.734127 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:44.734135 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:44.734809 (+ 674us) server_negotiation.cc:407] Connection header received
0809 00:59:44.734998 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:44.735002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:44.735078 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:44.735164 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:44.735978 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:44.735982 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:44.735985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:44.736000 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:44.736030 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:44.736042 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:44.736144 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:44.736184 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 00:59:44.736235 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:59:44.744323 9601 raft_consensus.cc:1261] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025 [term 1 FOLLOWER]: Refusing update from remote peer a6debb1192854880b0b340a3a5f5ab1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:44.744338 9880 raft_consensus.cc:1261] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767 [term 1 FOLLOWER]: Refusing update from remote peer a6debb1192854880b0b340a3a5f5ab1e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:44.745208 9958 consensus_queue.cc:1035] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b77606586d24bdbaaecbc885de5a025" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:44.745610 9955 consensus_queue.cc:1035] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e16b8c1397684a7db520a7bb7ec22767" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:44.756662 9961 mvcc.cc:205] Tried to move back new op lower bound from 7316427508703092736 to 7316427508558970880. Current Snapshot: MvccSnapshot[applied={T|T < 7316427508703092736}]
I20260809 00:59:44.770391 9448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:44.770634 9448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:44.774468 9448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e000f893edf4cd080929bebfb40c245: Sending DeleteTablet for 3 replicas of tablet 89d1116986854f95a4801156878cbb21
I20260809 00:59:44.775632 9580 tablet_service.cc:1514] Processing DeleteTablet for tablet 89d1116986854f95a4801156878cbb21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:44 UTC) from {username='slave'} at 127.0.0.1:41892
I20260809 00:59:44.775758 9859 tablet_service.cc:1514] Processing DeleteTablet for tablet 89d1116986854f95a4801156878cbb21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:44 UTC) from {username='slave'} at 127.0.0.1:40038
I20260809 00:59:44.776367 9721 tablet_service.cc:1514] Processing DeleteTablet for tablet 89d1116986854f95a4801156878cbb21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:44 UTC) from {username='slave'} at 127.0.0.1:42098
I20260809 00:59:44.776722 9972 tablet_replica.cc:330] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767: stopping tablet replica
I20260809 00:59:44.776922 9971 tablet_replica.cc:330] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025: stopping tablet replica
I20260809 00:59:44.777195 9972 raft_consensus.cc:2229] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:44.777307 9971 raft_consensus.cc:2229] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:44.777758 9972 raft_consensus.cc:2258] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:44.777801 9971 raft_consensus.cc:2258] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:44.779934 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 9513
I20260809 00:59:44.780046 9971 ts_tablet_manager.cc:1905] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:44.780762 9972 ts_tablet_manager.cc:1905] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:44.787803 9972 ts_tablet_manager.cc:1918] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:59:44.788028 9972 log.cc:1196] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/89d1116986854f95a4801156878cbb21
I20260809 00:59:44.788667 9972 ts_tablet_manager.cc:1939] T 89d1116986854f95a4801156878cbb21 P e16b8c1397684a7db520a7bb7ec22767: Deleting consensus metadata
I20260809 00:59:44.788769 9971 ts_tablet_manager.cc:1918] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:59:44.788944 9971 log.cc:1196] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/89d1116986854f95a4801156878cbb21
I20260809 00:59:44.789530 9971 ts_tablet_manager.cc:1939] T 89d1116986854f95a4801156878cbb21 P 4b77606586d24bdbaaecbc885de5a025: Deleting consensus metadata
I20260809 00:59:44.790709 9433 catalog_manager.cc:4841] TS e16b8c1397684a7db520a7bb7ec22767 (127.4.82.131:42845): tablet 89d1116986854f95a4801156878cbb21 (table test-table [id=c62a663b5cd5449297d1a2c1921f6746]) successfully deleted
I20260809 00:59:44.791339 9436 catalog_manager.cc:4841] TS 4b77606586d24bdbaaecbc885de5a025 (127.4.82.129:44223): tablet 89d1116986854f95a4801156878cbb21 (table test-table [id=c62a663b5cd5449297d1a2c1921f6746]) successfully deleted
I20260809 00:59:44.796386 9974 tablet_replica.cc:330] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e: stopping tablet replica
I20260809 00:59:44.796942 9974 raft_consensus.cc:2229] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:59:44.797565 9974 raft_consensus.cc:2258] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:44.798270 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.781082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41900 (local address 127.4.82.129:44223)
0809 00:59:44.781281 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:44.781288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:44.795855 (+ 14567us) server_negotiation.cc:407] Connection header received
0809 00:59:44.796108 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:44.796114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:44.796215 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:44.796274 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:44.797453 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:44.797459 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:44.797463 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:44.797482 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:44.797522 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:44.797535 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:44.797668 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:44.797847 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 00:59:44.797914 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 00:59:44.798952 9647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:44.799793 9974 ts_tablet_manager.cc:1905] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:44.807336 9974 ts_tablet_manager.cc:1918] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:59:44.807511 9974 log.cc:1196] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/89d1116986854f95a4801156878cbb21
I20260809 00:59:44.808087 9974 ts_tablet_manager.cc:1939] T 89d1116986854f95a4801156878cbb21 P a6debb1192854880b0b340a3a5f5ab1e: Deleting consensus metadata
I20260809 00:59:44.809913 9435 catalog_manager.cc:4841] TS a6debb1192854880b0b340a3a5f5ab1e (127.4.82.130:39901): tablet 89d1116986854f95a4801156878cbb21 (table test-table [id=c62a663b5cd5449297d1a2c1921f6746]) successfully deleted
I20260809 00:59:44.888114 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 9513
I20260809 00:59:44.913722 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 9656
I20260809 00:59:44.916790 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:44.914504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42104 (local address 127.4.82.130:39901)
0809 00:59:44.914701 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:44.914706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:44.914742 (+ 36us) server_negotiation.cc:407] Connection header received
0809 00:59:44.914896 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:44.914907 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:44.914972 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:44.915084 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:44.916085 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:44.916089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:44.916092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:44.916108 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:44.916144 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:44.916156 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:44.916276 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:44.916409 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 00:59:44.916462 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 00:59:44.917531 9786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:45.036073 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 9656
I20260809 00:59:45.054940 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 9794
I20260809 00:59:45.057571 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:45.055727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40052 (local address 127.4.82.131:42845)
0809 00:59:45.055906 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:45.055910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:45.055928 (+ 18us) server_negotiation.cc:407] Connection header received
0809 00:59:45.056050 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:45.056053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:45.056137 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:45.056221 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:45.056993 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:45.056997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:45.057000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:45.057013 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:45.057044 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:45.057055 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:45.057154 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:45.057218 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 00:59:45.057260 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 00:59:45.058205 9923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:45.145661 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 9794
I20260809 00:59:45.164326 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 9417
I20260809 00:59:45.165423 9481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:45.270334 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 9417
2026-08-09T00:59:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2729 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 00:59:45.313745 4426 mini_oidc.cc:152] Starting JWKS server
I20260809 00:59:45.316473 4426 webserver.cc:460] Webserver started at https://127.0.0.1:46691/ using document root <none> and password file <none>
I20260809 00:59:45.316838 4426 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 00:59:45.317508 4426 webserver.cc:460] Webserver started at http://127.0.0.1:43005/ using document root <none> and password file <none>
2026-08-09T00:59:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:45Z Disabled control of system clock
I20260809 00:59:45.347731 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:43107
--webserver_interface=127.4.82.190
--webserver_port=0
--builtin_ntp_servers=127.4.82.148:45583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:43107
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46691/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/testchainca.cert with env {}
W20260809 00:59:45.666690 9985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:45.666990 9985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:45.667171 9985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:45.675213 9985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:45.675386 9985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:45.675454 9985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:45.675534 9985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:45.675629 9985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:59:45.675715 9985 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:59:45.675791 9985 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46691/jwks.json
I20260809 00:59:45.685603 9985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:45583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:43107
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:43107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46691/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:45.686798 9985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:45.688130 9985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:45.699556 9993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:45.699497 9991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:45.699627 9990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:45.700346 9985 server_base.cc:1034] running on GCE node
I20260809 00:59:45.701049 9985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:45.702280 9985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:45.703461 9985 hybrid_clock.cc:648] HybridClock initialized: now 1786237185703428 us; error 37 us; skew 500 ppm
I20260809 00:59:45.703799 9985 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 00:59:45.706048 9985 webserver.cc:460] Webserver started at http://127.4.82.190:35969/ using document root <none> and password file <none>
I20260809 00:59:45.706704 9985 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:45.706830 9985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:45.707145 9985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:45.709942 9985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "5c19e28443764dfba8582dcaa6191e68"
format_stamp: "Formatted at 2026-08-09 00:59:45 on dist-test-slave-07c2"
server_key: "9b5b2ba0080582e5040f0015e4ceb1c8"
server_key_iv: "8543cd85c037d508f885679f5130546e"
server_key_version: "encryptionkey@0"
I20260809 00:59:45.710745 9985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "5c19e28443764dfba8582dcaa6191e68"
format_stamp: "Formatted at 2026-08-09 00:59:45 on dist-test-slave-07c2"
server_key: "9b5b2ba0080582e5040f0015e4ceb1c8"
server_key_iv: "8543cd85c037d508f885679f5130546e"
server_key_version: "encryptionkey@0"
I20260809 00:59:45.715732 9985 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 00:59:45.718909 9999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:45.720265 9985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 00:59:45.720402 9985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "5c19e28443764dfba8582dcaa6191e68"
format_stamp: "Formatted at 2026-08-09 00:59:45 on dist-test-slave-07c2"
server_key: "9b5b2ba0080582e5040f0015e4ceb1c8"
server_key_iv: "8543cd85c037d508f885679f5130546e"
server_key_version: "encryptionkey@0"
I20260809 00:59:45.720570 9985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:45.760831 9985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:45.761610 9985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:45.761885 9985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:45.779603 9985 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:43107
I20260809 00:59:45.779644 10055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:43107 every 8 connection(s)
I20260809 00:59:45.780900 9985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:45.784474 10056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:45.785915 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 9985
I20260809 00:59:45.786150 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:45.786988 4426 external_mini_cluster.cc:1428] Setting key b171018a222fa8cf2e252a3fcee49be2
I20260809 00:59:45.797174 10056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68: Bootstrap starting.
I20260809 00:59:45.799909 10059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:45.789012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55198 (local address 127.4.82.190:43107)
0809 00:59:45.789976 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:45.789986 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:45.790028 (+ 42us) server_negotiation.cc:407] Connection header received
0809 00:59:45.791632 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:45.791651 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:45.792097 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:45.792457 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:45.793051 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:45.794291 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:45.795666 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:45.796286 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:45.796483 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:45.797740 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:45.797747 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:45.797758 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:45.798253 (+ 495us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:45.798341 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:45.798362 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:45.798507 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:45.798725 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 00:59:45.798994 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":336,"threads_started":1}
I20260809 00:59:45.800396 10056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:45.801592 10056 log.cc:824] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:45.803757 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:59:45.804550 10056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68: No bootstrap required, opened a new log
I20260809 00:59:45.812238 10056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c19e28443764dfba8582dcaa6191e68" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 43107 } }
I20260809 00:59:45.812613 10056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:45.812724 10056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c19e28443764dfba8582dcaa6191e68, State: Initialized, Role: FOLLOWER
I20260809 00:59:45.813254 10056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c19e28443764dfba8582dcaa6191e68" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 43107 } }
I20260809 00:59:45.813477 10056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:45.813612 10056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:45.813784 10056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:45.815940 10056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c19e28443764dfba8582dcaa6191e68" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 43107 } }
I20260809 00:59:45.816500 10056 leader_election.cc:302] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c19e28443764dfba8582dcaa6191e68; no voters:
I20260809 00:59:45.817058 10056 leader_election.cc:288] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:45.817162 10061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:45.818288 10061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [term 1 LEADER]: Becoming Leader. State: Replica: 5c19e28443764dfba8582dcaa6191e68, State: Running, Role: LEADER
I20260809 00:59:45.818833 10061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c19e28443764dfba8582dcaa6191e68" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 43107 } }
I20260809 00:59:45.819156 10056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:45.821609 10063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c19e28443764dfba8582dcaa6191e68. Latest consensus state: current_term: 1 leader_uuid: "5c19e28443764dfba8582dcaa6191e68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c19e28443764dfba8582dcaa6191e68" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 43107 } } }
I20260809 00:59:45.822031 10063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:45.821934 10062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c19e28443764dfba8582dcaa6191e68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c19e28443764dfba8582dcaa6191e68" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 43107 } } }
I20260809 00:59:45.822203 10062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:45.823799 10070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:45.828342 10070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:45.836269 10070 catalog_manager.cc:1347] Generated new cluster ID: 502af36e9946403ea4e919529829cc32
I20260809 00:59:45.836366 10070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:45.852331 10070 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:45.854012 10070 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:45.862700 10070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c19e28443764dfba8582dcaa6191e68: Generated new TSK 0
I20260809 00:59:45.863600 10070 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 00:59:45.897470 10086 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 00:59:45.897676 10086 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 00:59:45.898824 10059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:45.876934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55206 (local address 127.4.82.190:43107)
0809 00:59:45.877165 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:45.877175 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:45.877775 (+ 600us) server_negotiation.cc:407] Connection header received
0809 00:59:45.878104 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:45.878110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:45.878200 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:45.878361 (+ 161us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:59:45.878848 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:45.879995 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:45.881433 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:45.882129 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:45.882265 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:45.882746 (+ 481us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:59:45.898182 (+ 15436us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46691/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 00:59:45.898529 (+ 347us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55206: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46691/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 00:59:45.898947 10059 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55206: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46691/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 00:59:45.900267 10085 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:59:45.876774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:43107 (local address 127.0.0.1:55206)
0809 00:59:45.877670 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:45.877685 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:45.877819 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:45.878379 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:45.878382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:45.878391 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:59:45.878695 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:45.878702 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:45.880130 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:45.880134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:45.881282 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:45.881294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:45.882313 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:45.882318 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:45.882440 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:45.882494 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:59:45.898426 (+ 15932us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46691/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 00:59:45.898686 (+ 260us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.82.190:43107: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46691/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":580,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.4.82.190:43107: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46691/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 00:59:45.901932 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 9985
I20260809 00:59:45.903049 10051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:45.994925 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 9985
2026-08-09T00:59:46Z chronyd exiting
I20260809 00:59:46.046787 4426 test_util.cc:175] -----------------------------------------------
I20260809 00:59:46.046973 4426 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786237144174446-4426-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (737 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 00:59:46.050750 4426 mini_oidc.cc:152] Starting JWKS server
I20260809 00:59:46.052685 4426 webserver.cc:460] Webserver started at https://127.0.0.1:33027/ using document root <none> and password file <none>
I20260809 00:59:46.052917 4426 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 00:59:46.053591 4426 webserver.cc:460] Webserver started at http://127.0.0.1:35511/ using document root <none> and password file <none>
2026-08-09T00:59:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:46Z Disabled control of system clock
I20260809 00:59:46.083716 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:33119
--webserver_interface=127.4.82.190
--webserver_port=0
--builtin_ntp_servers=127.4.82.148:35903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:33119
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33027/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/testchainca.cert with env {}
W20260809 00:59:46.397980 10093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:46.398239 10093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:46.398324 10093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:46.405221 10093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:46.405314 10093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:46.405351 10093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:46.405406 10093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:46.405442 10093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:59:46.405501 10093 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:59:46.405539 10093 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33027/jwks.json
I20260809 00:59:46.416102 10093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:33119
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:33119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33027/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:46.417213 10093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:46.418424 10093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:46.430245 10098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:46.430213 10102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:46.430155 10099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:46.431097 10093 server_base.cc:1034] running on GCE node
I20260809 00:59:46.431914 10093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:46.433215 10093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:46.434418 10093 hybrid_clock.cc:648] HybridClock initialized: now 1786237186434365 us; error 42 us; skew 500 ppm
I20260809 00:59:46.434765 10093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 00:59:46.441957 10093 webserver.cc:460] Webserver started at http://127.4.82.190:38155/ using document root <none> and password file <none>
I20260809 00:59:46.442639 10093 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:46.442770 10093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:46.443105 10093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:46.445931 10093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "65548eee5dc74aa79cd00f1890bb4503"
format_stamp: "Formatted at 2026-08-09 00:59:46 on dist-test-slave-07c2"
server_key: "3e8f7a94b8331d1255c45fde89073f8a"
server_key_iv: "686ffbe879d5b1ea7b0c0d11c07bd946"
server_key_version: "encryptionkey@0"
I20260809 00:59:46.446761 10093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "65548eee5dc74aa79cd00f1890bb4503"
format_stamp: "Formatted at 2026-08-09 00:59:46 on dist-test-slave-07c2"
server_key: "3e8f7a94b8331d1255c45fde89073f8a"
server_key_iv: "686ffbe879d5b1ea7b0c0d11c07bd946"
server_key_version: "encryptionkey@0"
I20260809 00:59:46.451642 10093 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.005s
I20260809 00:59:46.454838 10109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:46.456113 10093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 00:59:46.456292 10093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "65548eee5dc74aa79cd00f1890bb4503"
format_stamp: "Formatted at 2026-08-09 00:59:46 on dist-test-slave-07c2"
server_key: "3e8f7a94b8331d1255c45fde89073f8a"
server_key_iv: "686ffbe879d5b1ea7b0c0d11c07bd946"
server_key_version: "encryptionkey@0"
I20260809 00:59:46.456496 10093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:46.476445 10093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:46.477200 10093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:46.477504 10093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:46.495783 10093 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:33119
I20260809 00:59:46.495817 10161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:33119 every 8 connection(s)
I20260809 00:59:46.497017 10093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:46.500530 10162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:46.500631 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 10093
I20260809 00:59:46.500841 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:46.501479 4426 external_mini_cluster.cc:1428] Setting key 14a550be921937387fee75f4a32d15a0
I20260809 00:59:46.514933 10162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503: Bootstrap starting.
I20260809 00:59:46.517035 10165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:46.503468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52060 (local address 127.4.82.190:33119)
0809 00:59:46.504563 (+ 1095us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:46.504579 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:46.504631 (+ 52us) server_negotiation.cc:407] Connection header received
0809 00:59:46.507159 (+ 2528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:46.507187 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:46.507772 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:46.508204 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:46.508740 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:46.510607 (+ 1867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:46.511901 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:46.512990 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:46.513158 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:46.514253 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:46.514265 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:46.514285 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:46.514985 (+ 700us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:46.515083 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:46.515108 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:46.515238 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:46.515479 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 00:59:46.515781 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":360,"threads_started":1}
I20260809 00:59:46.518563 10162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:46.519788 10162 log.cc:824] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:46.520794 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:59:46.522452 10162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503: No bootstrap required, opened a new log
I20260809 00:59:46.529918 10162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65548eee5dc74aa79cd00f1890bb4503" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33119 } }
I20260809 00:59:46.530264 10162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:46.530373 10162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65548eee5dc74aa79cd00f1890bb4503, State: Initialized, Role: FOLLOWER
I20260809 00:59:46.530948 10162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65548eee5dc74aa79cd00f1890bb4503" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33119 } }
I20260809 00:59:46.531198 10162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:46.531399 10162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:46.531560 10162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:46.533555 10162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65548eee5dc74aa79cd00f1890bb4503" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33119 } }
I20260809 00:59:46.534003 10162 leader_election.cc:302] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65548eee5dc74aa79cd00f1890bb4503; no voters:
I20260809 00:59:46.534531 10162 leader_election.cc:288] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:46.534614 10167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:46.535449 10167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [term 1 LEADER]: Becoming Leader. State: Replica: 65548eee5dc74aa79cd00f1890bb4503, State: Running, Role: LEADER
I20260809 00:59:46.536110 10167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65548eee5dc74aa79cd00f1890bb4503" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33119 } }
I20260809 00:59:46.536825 10162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:46.539052 10168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65548eee5dc74aa79cd00f1890bb4503" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65548eee5dc74aa79cd00f1890bb4503" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33119 } } }
I20260809 00:59:46.539539 10168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:46.539024 10169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65548eee5dc74aa79cd00f1890bb4503. Latest consensus state: current_term: 1 leader_uuid: "65548eee5dc74aa79cd00f1890bb4503" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65548eee5dc74aa79cd00f1890bb4503" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33119 } } }
I20260809 00:59:46.540149 10169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:46.541731 10172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:46.546675 10172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:46.556557 10172 catalog_manager.cc:1347] Generated new cluster ID: 0e09c8b097d04b8cbaf6f3eb03e6207a
I20260809 00:59:46.556679 10172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:46.568492 10172 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:46.569736 10172 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:46.576395 10172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65548eee5dc74aa79cd00f1890bb4503: Generated new TSK 0
I20260809 00:59:46.577225 10172 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 00:59:46.613989 10195 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 00:59:46.614108 10195 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 00:59:46.615092 10165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:46.595626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52070 (local address 127.4.82.190:33119)
0809 00:59:46.595876 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:46.595884 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:46.596256 (+ 372us) server_negotiation.cc:407] Connection header received
0809 00:59:46.596496 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:46.596500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:46.596569 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:46.596684 (+ 115us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:59:46.597264 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:46.597984 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:46.600021 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:46.600447 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:46.600535 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:46.601102 (+ 567us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:59:46.614600 (+ 13498us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 00:59:46.614876 (+ 276us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52070: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 00:59:46.615252 10165 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52070: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 00:59:46.615497 10194 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:59:46.595500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33119 (local address 127.0.0.1:52070)
0809 00:59:46.596151 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:46.596172 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:46.596327 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:46.596737 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:46.596743 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:46.596769 (+ 26us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:59:46.597073 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:46.597104 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:46.598129 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:46.598135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:46.599888 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:46.599904 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:46.600611 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:46.600618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:46.600826 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:46.600871 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:59:46.614802 (+ 13931us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 00:59:46.615051 (+ 249us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.82.190:33119: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33027/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":340,"threads_started":1}
I20260809 00:59:46.616447 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 10093
I20260809 00:59:46.617540 10157 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:46.707154 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 10093
2026-08-09T00:59:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (712 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 00:59:46.763772 4426 mini_oidc.cc:152] Starting JWKS server
I20260809 00:59:46.765684 4426 webserver.cc:460] Webserver started at https://127.0.0.1:42651/ using document root <none> and password file <none>
I20260809 00:59:46.765890 4426 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 00:59:46.766476 4426 webserver.cc:460] Webserver started at http://127.0.0.1:40615/ using document root <none> and password file <none>
2026-08-09T00:59:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:46Z Disabled control of system clock
I20260809 00:59:46.798038 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:38435
--webserver_interface=127.4.82.190
--webserver_port=0
--builtin_ntp_servers=127.4.82.148:39133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:38435
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42651/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 00:59:47.122213 10202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:47.122565 10202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:47.122727 10202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:47.130012 10202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:47.130159 10202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:47.130260 10202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:47.130381 10202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:47.130462 10202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:59:47.130554 10202 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:59:47.130643 10202 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42651/jwks.json
I20260809 00:59:47.139951 10202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:39133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:38435
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:38435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42651/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:47.141291 10202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:47.143097 10202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:47.155169 10210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:47.155248 10208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:47.155218 10207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:47.156219 10202 server_base.cc:1034] running on GCE node
I20260809 00:59:47.157051 10202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:47.158461 10202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:47.159655 10202 hybrid_clock.cc:648] HybridClock initialized: now 1786237187159595 us; error 51 us; skew 500 ppm
I20260809 00:59:47.160003 10202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 00:59:47.162285 10202 webserver.cc:460] Webserver started at http://127.4.82.190:41161/ using document root <none> and password file <none>
I20260809 00:59:47.162969 10202 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:47.163089 10202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:47.163501 10202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:47.166395 10202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "b932c21c796c482ca8a920a0b621eeb7"
format_stamp: "Formatted at 2026-08-09 00:59:47 on dist-test-slave-07c2"
server_key: "6bce10cce6387ce3893246bd83b84ae5"
server_key_iv: "1cb13771aa26e0e4910b494d1a18ed4e"
server_key_version: "encryptionkey@0"
I20260809 00:59:47.167425 10202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "b932c21c796c482ca8a920a0b621eeb7"
format_stamp: "Formatted at 2026-08-09 00:59:47 on dist-test-slave-07c2"
server_key: "6bce10cce6387ce3893246bd83b84ae5"
server_key_iv: "1cb13771aa26e0e4910b494d1a18ed4e"
server_key_version: "encryptionkey@0"
I20260809 00:59:47.172428 10202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 00:59:47.175863 10217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:47.177134 10202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 00:59:47.177322 10202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "b932c21c796c482ca8a920a0b621eeb7"
format_stamp: "Formatted at 2026-08-09 00:59:47 on dist-test-slave-07c2"
server_key: "6bce10cce6387ce3893246bd83b84ae5"
server_key_iv: "1cb13771aa26e0e4910b494d1a18ed4e"
server_key_version: "encryptionkey@0"
I20260809 00:59:47.177597 10202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:47.194751 10202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:47.195683 10202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:47.196012 10202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:47.219419 10202 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:38435
I20260809 00:59:47.219436 10270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:38435 every 8 connection(s)
I20260809 00:59:47.220806 10202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:47.224470 10271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:47.228790 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 10202
I20260809 00:59:47.229022 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:47.229856 4426 external_mini_cluster.cc:1428] Setting key 41e43ae6cc1256c9a3186c97a99260cf
I20260809 00:59:47.236168 10271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7: Bootstrap starting.
I20260809 00:59:47.239141 10271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:47.240235 10271 log.cc:824] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:47.243734 10271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7: No bootstrap required, opened a new log
I20260809 00:59:47.243932 10274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:47.231819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38414 (local address 127.4.82.190:38435)
0809 00:59:47.232793 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:47.232804 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:47.232843 (+ 39us) server_negotiation.cc:407] Connection header received
0809 00:59:47.234784 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:47.234806 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:47.235254 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:47.235704 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:47.236250 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:47.237627 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:47.238930 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:47.239732 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:47.239944 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:47.241220 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:47.241230 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:47.241242 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:47.241901 (+ 659us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:47.242004 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:47.242030 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:47.242156 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:47.242361 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 00:59:47.242678 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":366,"threads_started":1}
I20260809 00:59:47.247623 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:59:47.251950 10271 raft_consensus.cc:348] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b932c21c796c482ca8a920a0b621eeb7" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38435 } }
I20260809 00:59:47.252315 10271 raft_consensus.cc:374] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:47.252426 10271 raft_consensus.cc:729] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b932c21c796c482ca8a920a0b621eeb7, State: Initialized, Role: FOLLOWER
I20260809 00:59:47.253007 10271 consensus_queue.cc:260] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b932c21c796c482ca8a920a0b621eeb7" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38435 } }
I20260809 00:59:47.253257 10271 raft_consensus.cc:388] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:47.253401 10271 raft_consensus.cc:482] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:47.253582 10271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:47.255822 10271 raft_consensus.cc:504] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b932c21c796c482ca8a920a0b621eeb7" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38435 } }
I20260809 00:59:47.256404 10271 leader_election.cc:302] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b932c21c796c482ca8a920a0b621eeb7; no voters:
I20260809 00:59:47.257057 10271 leader_election.cc:288] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:47.257185 10276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:47.257912 10276 raft_consensus.cc:686] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [term 1 LEADER]: Becoming Leader. State: Replica: b932c21c796c482ca8a920a0b621eeb7, State: Running, Role: LEADER
I20260809 00:59:47.258401 10276 consensus_queue.cc:237] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b932c21c796c482ca8a920a0b621eeb7" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38435 } }
I20260809 00:59:47.259071 10271 sys_catalog.cc:564] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:47.261188 10278 sys_catalog.cc:455] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b932c21c796c482ca8a920a0b621eeb7. Latest consensus state: current_term: 1 leader_uuid: "b932c21c796c482ca8a920a0b621eeb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b932c21c796c482ca8a920a0b621eeb7" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38435 } } }
I20260809 00:59:47.261605 10278 sys_catalog.cc:458] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:47.261497 10277 sys_catalog.cc:455] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b932c21c796c482ca8a920a0b621eeb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b932c21c796c482ca8a920a0b621eeb7" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38435 } } }
I20260809 00:59:47.261821 10277 sys_catalog.cc:458] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:47.263592 10287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:47.269618 10287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:47.277874 10287 catalog_manager.cc:1347] Generated new cluster ID: bf2af38ee7a24023aad48d0ee53d6954
I20260809 00:59:47.277993 10287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:47.291698 10287 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:47.292925 10287 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:47.299553 10287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b932c21c796c482ca8a920a0b621eeb7: Generated new TSK 0
I20260809 00:59:47.300329 10287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:59:47.323633 10274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:47.315317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38420 (local address 127.4.82.190:38435)
0809 00:59:47.315524 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:47.315529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:47.315856 (+ 327us) server_negotiation.cc:407] Connection header received
0809 00:59:47.316105 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:47.316109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:47.316175 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:47.316304 (+ 129us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:59:47.316825 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:47.317540 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:47.319609 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:47.320014 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:47.320107 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:47.320677 (+ 570us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:59:47.322983 (+ 2306us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42651/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 00:59:47.323323 (+ 340us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38420: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42651/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":77}
W20260809 00:59:47.323748 10274 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38420: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42651/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 00:59:47.325736 10303 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:59:47.315088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:38435 (local address 127.0.0.1:38420)
0809 00:59:47.315747 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:47.315767 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:47.315921 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:47.316354 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:47.316359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:47.316372 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:59:47.316674 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:47.316687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:47.317691 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:47.317698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:47.319472 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:47.319488 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:47.320212 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:47.320219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:47.320417 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:47.320463 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:59:47.323207 (+ 2744us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42651/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 00:59:47.325277 (+ 2070us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.82.190:38435: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42651/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":1735,"thread_start_us":350,"threads_started":1}
I20260809 00:59:47.326773 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 10202
I20260809 00:59:47.327865 10266 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 00:59:47.335846 10304 webserver.cc:533] Webserver: sslize failed:
E20260809 00:59:47.335961 10304 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 00:59:47.419196 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 10202
2026-08-09T00:59:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (712 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T00:59:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:47Z Disabled control of system clock
I20260809 00:59:47.505429 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:44971
--webserver_interface=127.4.82.190
--webserver_port=0
--builtin_ntp_servers=127.4.82.148:39591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:44971
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/testchainca.cert with env {}
W20260809 00:59:47.830667 10309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:47.830941 10309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:47.831054 10309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:47.838125 10309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:47.838219 10309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:47.838304 10309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:47.838407 10309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:47.838474 10309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:59:47.838595 10309 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:59:47.838642 10309 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 00:59:47.848147 10309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:39591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:44971
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:44971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:47.849385 10309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:47.850636 10309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:47.862099 10315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:47.862170 10317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:47.862131 10314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:47.862949 10309 server_base.cc:1034] running on GCE node
I20260809 00:59:47.863718 10309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:47.865012 10309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:47.866235 10309 hybrid_clock.cc:648] HybridClock initialized: now 1786237187866195 us; error 52 us; skew 500 ppm
I20260809 00:59:47.866571 10309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 00:59:47.868870 10309 webserver.cc:460] Webserver started at http://127.4.82.190:42643/ using document root <none> and password file <none>
I20260809 00:59:47.869531 10309 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:47.869675 10309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:47.869997 10309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:47.872871 10309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "b23916bc1087472996009980de20ab96"
format_stamp: "Formatted at 2026-08-09 00:59:47 on dist-test-slave-07c2"
server_key: "0bae6e71eee29b459d784b7a93d44452"
server_key_iv: "81e944a4a8cf541ff87eb3d607381209"
server_key_version: "encryptionkey@0"
I20260809 00:59:47.873669 10309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "b23916bc1087472996009980de20ab96"
format_stamp: "Formatted at 2026-08-09 00:59:47 on dist-test-slave-07c2"
server_key: "0bae6e71eee29b459d784b7a93d44452"
server_key_iv: "81e944a4a8cf541ff87eb3d607381209"
server_key_version: "encryptionkey@0"
I20260809 00:59:47.878485 10309 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 00:59:47.881637 10323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:47.882900 10309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 00:59:47.883087 10309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "b23916bc1087472996009980de20ab96"
format_stamp: "Formatted at 2026-08-09 00:59:47 on dist-test-slave-07c2"
server_key: "0bae6e71eee29b459d784b7a93d44452"
server_key_iv: "81e944a4a8cf541ff87eb3d607381209"
server_key_version: "encryptionkey@0"
I20260809 00:59:47.883345 10309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:47.900189 10309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:47.900915 10309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:47.901188 10309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:47.919028 10309 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:44971
I20260809 00:59:47.919059 10376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:44971 every 8 connection(s)
I20260809 00:59:47.920238 10309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:47.923101 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 10309
I20260809 00:59:47.923326 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:47.923745 10377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:47.924021 4426 external_mini_cluster.cc:1428] Setting key 2184445bc4c8b16fb7526150b9fe6e78
I20260809 00:59:47.938269 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:47.925800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.4.82.190:44971)
0809 00:59:47.926874 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:47.926887 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:47.926934 (+ 47us) server_negotiation.cc:407] Connection header received
0809 00:59:47.928581 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:47.928618 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:47.929166 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:47.929572 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:47.930064 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:47.932062 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:47.933330 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:47.934347 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:47.934533 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:47.935690 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:47.935700 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:47.935715 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:59:47.936406 (+ 691us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:47.936490 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:59:47.936518 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:47.936624 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:47.936816 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 00:59:47.937102 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":362,"threads_started":1}
I20260809 00:59:47.938298 10377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96: Bootstrap starting.
I20260809 00:59:47.941412 10377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:47.941972 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:59:47.942601 10377 log.cc:824] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:47.945549 10377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96: No bootstrap required, opened a new log
I20260809 00:59:47.952477 10377 raft_consensus.cc:348] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23916bc1087472996009980de20ab96" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44971 } }
I20260809 00:59:47.952773 10377 raft_consensus.cc:374] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:47.952849 10377 raft_consensus.cc:729] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b23916bc1087472996009980de20ab96, State: Initialized, Role: FOLLOWER
I20260809 00:59:47.953384 10377 consensus_queue.cc:260] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23916bc1087472996009980de20ab96" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44971 } }
I20260809 00:59:47.953660 10377 raft_consensus.cc:388] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:47.953806 10377 raft_consensus.cc:482] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:47.953980 10377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:47.956226 10377 raft_consensus.cc:504] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23916bc1087472996009980de20ab96" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44971 } }
I20260809 00:59:47.956691 10377 leader_election.cc:302] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b23916bc1087472996009980de20ab96; no voters:
I20260809 00:59:47.957228 10377 leader_election.cc:288] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:47.957379 10383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:47.958226 10383 raft_consensus.cc:686] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [term 1 LEADER]: Becoming Leader. State: Replica: b23916bc1087472996009980de20ab96, State: Running, Role: LEADER
I20260809 00:59:47.958779 10383 consensus_queue.cc:237] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23916bc1087472996009980de20ab96" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44971 } }
I20260809 00:59:47.959436 10377 sys_catalog.cc:564] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:47.961452 10384 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b23916bc1087472996009980de20ab96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23916bc1087472996009980de20ab96" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44971 } } }
I20260809 00:59:47.961771 10384 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:47.963258 10389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:47.963302 10385 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b23916bc1087472996009980de20ab96. Latest consensus state: current_term: 1 leader_uuid: "b23916bc1087472996009980de20ab96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23916bc1087472996009980de20ab96" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44971 } } }
I20260809 00:59:47.963631 10385 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96 [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:47.968169 10389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:47.977941 10389 catalog_manager.cc:1347] Generated new cluster ID: 49605405445142ad86641c42d33e58c2
I20260809 00:59:47.978081 10389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:47.995534 10389 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:47.996649 10389 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:48.001928 10389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b23916bc1087472996009980de20ab96: Generated new TSK 0
I20260809 00:59:48.002792 10389 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:59:48.048720 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:48.014400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34004 (local address 127.4.82.190:44971)
0809 00:59:48.014545 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:48.014550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:48.014951 (+ 401us) server_negotiation.cc:407] Connection header received
0809 00:59:48.015192 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:48.015196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:48.015271 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:48.015444 (+ 173us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:59:48.016020 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:48.016725 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:48.018728 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:48.019124 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:48.019220 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:48.019802 (+ 582us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:59:48.048193 (+ 28391us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 00:59:48.048466 (+ 273us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34004: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 00:59:48.048879 10381 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34004: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 00:59:48.049118 10408 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:59:48.014225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44971 (local address 127.0.0.1:34004)
0809 00:59:48.014840 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:48.014864 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:48.015015 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:48.015494 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:48.015499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:48.015514 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:59:48.015872 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:48.015884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:48.016878 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:48.016884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:48.018599 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:48.018613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:48.019321 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:48.019330 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:48.019508 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:48.019556 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:59:48.048414 (+ 28858us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 00:59:48.048718 (+ 304us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.82.190:44971: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":327,"threads_started":1}
I20260809 00:59:48.050063 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 10309
I20260809 00:59:48.051149 10372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:48.143100 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 10309
2026-08-09T00:59:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (703 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (401 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (403 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:59:49 dist-test-slave-07c2 krb5kdc[10425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:59:49 dist-test-slave-07c2 krb5kdc[10425](info): set up 2 sockets
Aug 09 00:59:49 dist-test-slave-07c2 krb5kdc[10425](info): commencing operation
krb5kdc: starting...
W20260809 00:59:50.074863 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 00:59:50 dist-test-slave-07c2 krb5kdc[10425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:59:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 00:59:50.282742 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:44169
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:38777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:44169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:50.598740 10439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:50.599071 10439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:50.599246 10439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:50.606320 10439 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:50.606443 10439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:50.606550 10439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:50.606653 10439 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:50.606736 10439 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:50.616309 10439 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:38777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:44169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:44169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:50.617514 10439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:50.618716 10439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:50.629981 10447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:50.630054 10444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:50.630084 10445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:50.631187 10439 server_base.cc:1034] running on GCE node
I20260809 00:59:50.631975 10439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:50.633204 10439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:50.634446 10439 hybrid_clock.cc:648] HybridClock initialized: now 1786237190634406 us; error 46 us; skew 500 ppm
I20260809 00:59:50.634783 10439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:50 dist-test-slave-07c2 krb5kdc[10425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237190, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:50.637864 10439 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 00:59:50.639200 10439 webserver.cc:460] Webserver started at http://127.4.82.190:38357/ using document root <none> and password file <none>
I20260809 00:59:50.639880 10439 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:50.640031 10439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:50.640341 10439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:50.643119 10439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "44952ef163d64cb1aa4027ae7dc2d36d"
format_stamp: "Formatted at 2026-08-09 00:59:50 on dist-test-slave-07c2"
server_key: "325a6ab667fa4fff77eca0f366326b60"
server_key_iv: "fd7d7f9ca9750d7aa20c3fdca0627335"
server_key_version: "encryptionkey@0"
I20260809 00:59:50.643949 10439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "44952ef163d64cb1aa4027ae7dc2d36d"
format_stamp: "Formatted at 2026-08-09 00:59:50 on dist-test-slave-07c2"
server_key: "325a6ab667fa4fff77eca0f366326b60"
server_key_iv: "fd7d7f9ca9750d7aa20c3fdca0627335"
server_key_version: "encryptionkey@0"
I20260809 00:59:50.648689 10439 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 00:59:50.651865 10454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:50.653043 10439 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:59:50.653231 10439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "44952ef163d64cb1aa4027ae7dc2d36d"
format_stamp: "Formatted at 2026-08-09 00:59:50 on dist-test-slave-07c2"
server_key: "325a6ab667fa4fff77eca0f366326b60"
server_key_iv: "fd7d7f9ca9750d7aa20c3fdca0627335"
server_key_version: "encryptionkey@0"
I20260809 00:59:50.653429 10439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:50.682293 10439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:50.686138 10439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:50.686434 10439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:50.704913 10439 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:44169
I20260809 00:59:50.704950 10508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:44169 every 8 connection(s)
I20260809 00:59:50.706205 10439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:50.709780 10509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:50.711818 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 10439
I20260809 00:59:50.712028 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:50.712852 4426 external_mini_cluster.cc:1428] Setting key 1870409c4dd065d55dc68ad94c18414a
I20260809 00:59:50.722313 10509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d: Bootstrap starting.
I20260809 00:59:50.726171 10509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:50.727392 10509 log.cc:824] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d: Log is configured to *not* fsync() on all Append() calls
Aug 09 00:59:50 dist-test-slave-07c2 krb5kdc[10425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:50.730149 10509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d: No bootstrap required, opened a new log
I20260809 00:59:50.735707 10512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:50.714881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40704 (local address 127.4.82.190:44169)
0809 00:59:50.715691 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:50.715705 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:50.715750 (+ 45us) server_negotiation.cc:407] Connection header received
0809 00:59:50.717390 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:50.717410 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:50.717773 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:50.718051 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:50.719849 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:50.721135 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:50.722906 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:50.723595 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:50.723738 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:50.728567 (+ 4829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:50.728575 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:50.728588 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:50.728632 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:50.731684 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:50.732399 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:50.732414 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:50.732421 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:50.732591 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:50.733013 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:50.733020 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:50.733027 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:50.733529 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:50.733841 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:50.734287 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 00:59:50.734678 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":389,"threads_started":1}
I20260809 00:59:50.737835 10509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44952ef163d64cb1aa4027ae7dc2d36d" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44169 } }
I20260809 00:59:50.738183 10509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:50.738300 10509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44952ef163d64cb1aa4027ae7dc2d36d, State: Initialized, Role: FOLLOWER
I20260809 00:59:50.738806 10509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44952ef163d64cb1aa4027ae7dc2d36d" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44169 } }
I20260809 00:59:50.739007 10509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:50.739106 10509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:50.739229 10509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:50.741590 10509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44952ef163d64cb1aa4027ae7dc2d36d" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44169 } }
I20260809 00:59:50.741991 10509 leader_election.cc:302] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44952ef163d64cb1aa4027ae7dc2d36d; no voters:
I20260809 00:59:50.742492 10509 leader_election.cc:288] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:50.742631 10514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:50.743403 10514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [term 1 LEADER]: Becoming Leader. State: Replica: 44952ef163d64cb1aa4027ae7dc2d36d, State: Running, Role: LEADER
I20260809 00:59:50.743989 10514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44952ef163d64cb1aa4027ae7dc2d36d" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44169 } }
I20260809 00:59:50.744695 10509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:50.747229 10517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44952ef163d64cb1aa4027ae7dc2d36d. Latest consensus state: current_term: 1 leader_uuid: "44952ef163d64cb1aa4027ae7dc2d36d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44952ef163d64cb1aa4027ae7dc2d36d" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44169 } } }
I20260809 00:59:50.747267 10515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44952ef163d64cb1aa4027ae7dc2d36d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44952ef163d64cb1aa4027ae7dc2d36d" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44169 } } }
I20260809 00:59:50.747653 10517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:50.747695 10515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:50.752389 10523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:50.757115 10523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:50.765127 10523 catalog_manager.cc:1347] Generated new cluster ID: 5c9a880deb3b42c1b83bb38f2865eea6
I20260809 00:59:50.765234 10523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:50.777168 10523 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:50.778995 10523 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:50.794385 10523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44952ef163d64cb1aa4027ae7dc2d36d: Generated new TSK 0
I20260809 00:59:50.795389 10523 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 00:59:50.848699 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44169
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:38777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:51.169879 10539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:51.170132 10539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:51.170217 10539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:51.177060 10539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:51.177148 10539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:51.177250 10539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 00:59:51.186563 10539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:38777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:51.187726 10539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:51.188980 10539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:51.201289 10544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:51.201409 10545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:51.201448 10548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:51.202381 10539 server_base.cc:1034] running on GCE node
I20260809 00:59:51.202931 10539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:51.203830 10539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:51.205049 10539 hybrid_clock.cc:648] HybridClock initialized: now 1786237191205014 us; error 48 us; skew 500 ppm
I20260809 00:59:51.205338 10539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:51 dist-test-slave-07c2 krb5kdc[10425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237191, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:51.208318 10539 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 00:59:51.209708 10539 webserver.cc:460] Webserver started at http://127.4.82.129:43689/ using document root <none> and password file <none>
I20260809 00:59:51.210323 10539 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:51.210467 10539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:51.210764 10539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:51.213479 10539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "eabc947b4b544439a9383047ad752d7b"
format_stamp: "Formatted at 2026-08-09 00:59:51 on dist-test-slave-07c2"
server_key: "4cf9694d56b77769debc508a77cf4fab"
server_key_iv: "a49cc04fef18786d3931f97bcb3e268e"
server_key_version: "encryptionkey@0"
I20260809 00:59:51.214236 10539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "eabc947b4b544439a9383047ad752d7b"
format_stamp: "Formatted at 2026-08-09 00:59:51 on dist-test-slave-07c2"
server_key: "4cf9694d56b77769debc508a77cf4fab"
server_key_iv: "a49cc04fef18786d3931f97bcb3e268e"
server_key_version: "encryptionkey@0"
I20260809 00:59:51.219167 10539 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 00:59:51.222375 10555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:51.223637 10539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 00:59:51.223803 10539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "eabc947b4b544439a9383047ad752d7b"
format_stamp: "Formatted at 2026-08-09 00:59:51 on dist-test-slave-07c2"
server_key: "4cf9694d56b77769debc508a77cf4fab"
server_key_iv: "a49cc04fef18786d3931f97bcb3e268e"
server_key_version: "encryptionkey@0"
I20260809 00:59:51.223989 10539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:51.258064 10539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:51.261853 10539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:51.262164 10539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:51.263064 10539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:51.264333 10539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:51.264433 10539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:51.264612 10539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:51.264694 10539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:51.297241 10539 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:32845
I20260809 00:59:51.297264 10683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:32845 every 8 connection(s)
I20260809 00:59:51.298485 10539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 00:59:51.300122 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 10539
I20260809 00:59:51.300348 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 00:59:51.301096 4426 external_mini_cluster.cc:1428] Setting key 66d343677c9d5d43f4967aa05de56581
Aug 09 00:59:51 dist-test-slave-07c2 krb5kdc[10425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237191, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:51.319099 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:51.301192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:36813 (local address 127.4.82.190:44169)
0809 00:59:51.301675 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:51.301681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:51.302830 (+ 1149us) server_negotiation.cc:407] Connection header received
0809 00:59:51.304695 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:51.304704 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:51.304792 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:51.304922 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:51.306252 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:51.307357 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:51.308803 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:51.309518 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:51.309659 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:51.313287 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:51.313292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:51.313295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:51.313324 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:51.315995 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:51.316880 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:51.316885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:51.316887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:51.316997 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:51.317501 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:51.317505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:51.317507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:51.317744 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:51.317849 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:51.318680 (+ 831us) server_negotiation.cc:299] Negotiation successful
0809 00:59:51.318832 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":236,"threads_started":1}
I20260809 00:59:51.320621 10687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:51.301788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44169 (local address 127.4.82.129:36813)
0809 00:59:51.302646 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:51.302690 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:51.304442 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:51.305103 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:51.305111 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:51.305578 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:51.306092 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:51.306108 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:51.307479 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:51.307483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:51.308655 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:51.308666 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:51.309730 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:51.309735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:51.309918 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:51.310598 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:51.310624 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:51.313090 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:51.316129 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:51.316140 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:51.316156 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:51.316768 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:51.317104 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:51.317112 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:51.317118 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:51.317388 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:51.317870 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:51.317880 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:51.318299 (+ 419us) client_negotiation.cc:769] Sending connection context
0809 00:59:51.318606 (+ 307us) client_negotiation.cc:240] Negotiation successful
0809 00:59:51.319195 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":408,"threads_started":1}
I20260809 00:59:51.322259 10684 heartbeater.cc:347] Connected to a master server at 127.4.82.190:44169
I20260809 00:59:51.322571 10684 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:51.323356 10684 heartbeater.cc:511] Master 127.4.82.190:44169 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
I20260809 00:59:51.325428 10471 ts_manager.cc:194] Registered new tserver with Master: eabc947b4b544439a9383047ad752d7b (127.4.82.129:32845)
I20260809 00:59:51.327096 10471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:36813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 00:59:51.352725 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44169
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:38777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:51.678156 10690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:51.678426 10690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:51.678545 10690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:51.685691 10690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:51.685808 10690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:51.686000 10690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 00:59:51.695343 10690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:38777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:51.696486 10690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:51.697803 10690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:51.710208 10696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:51.710237 10695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:51.710750 10698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:51.711218 10690 server_base.cc:1034] running on GCE node
I20260809 00:59:51.711802 10690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:51.712658 10690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:51.713886 10690 hybrid_clock.cc:648] HybridClock initialized: now 1786237191713837 us; error 40 us; skew 500 ppm
I20260809 00:59:51.714175 10690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:51 dist-test-slave-07c2 krb5kdc[10425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237191, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:51.719892 10690 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 00:59:51.721350 10690 webserver.cc:460] Webserver started at http://127.4.82.130:46481/ using document root <none> and password file <none>
I20260809 00:59:51.722126 10690 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:51.722260 10690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:51.722555 10690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:51.725277 10690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "8010995d16e944ae8fe3366f75f0702f"
format_stamp: "Formatted at 2026-08-09 00:59:51 on dist-test-slave-07c2"
server_key: "bb8b5c4f2cb5be66107fccf60009ab46"
server_key_iv: "fa651738881dec75df8cc91896fd8ec8"
server_key_version: "encryptionkey@0"
I20260809 00:59:51.726017 10690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "8010995d16e944ae8fe3366f75f0702f"
format_stamp: "Formatted at 2026-08-09 00:59:51 on dist-test-slave-07c2"
server_key: "bb8b5c4f2cb5be66107fccf60009ab46"
server_key_iv: "fa651738881dec75df8cc91896fd8ec8"
server_key_version: "encryptionkey@0"
I20260809 00:59:51.731097 10690 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 00:59:51.734316 10705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:51.735579 10690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 00:59:51.735739 10690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "8010995d16e944ae8fe3366f75f0702f"
format_stamp: "Formatted at 2026-08-09 00:59:51 on dist-test-slave-07c2"
server_key: "bb8b5c4f2cb5be66107fccf60009ab46"
server_key_iv: "fa651738881dec75df8cc91896fd8ec8"
server_key_version: "encryptionkey@0"
I20260809 00:59:51.735931 10690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:51.750548 10690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:51.754278 10690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:51.754552 10690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:51.755491 10690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:51.756796 10690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:51.756912 10690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:51.757062 10690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:51.757133 10690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:51.789906 10690 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:33203
I20260809 00:59:51.789981 10825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:33203 every 8 connection(s)
I20260809 00:59:51.791121 10690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 00:59:51.792246 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 10690
I20260809 00:59:51.792420 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 00:59:51.792999 4426 external_mini_cluster.cc:1428] Setting key 91a17665069f944c3a55e6dc2a23816c
Aug 09 00:59:51 dist-test-slave-07c2 krb5kdc[10425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237191, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:51.812183 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:51.793821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:55969 (local address 127.4.82.190:44169)
0809 00:59:51.794037 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:51.794044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:51.795327 (+ 1283us) server_negotiation.cc:407] Connection header received
0809 00:59:51.797272 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:51.797281 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:51.797373 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:51.797484 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:51.798833 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:51.800009 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:51.802187 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:51.802963 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:51.803108 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:51.806618 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:51.806622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:51.806626 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:51.806659 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:51.809238 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:51.809922 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:51.809929 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:51.809932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:51.810069 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:51.810510 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:51.810515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:51.810518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:51.810777 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:51.810945 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:51.811620 (+ 675us) server_negotiation.cc:299] Negotiation successful
0809 00:59:51.811846 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 00:59:51.813093 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:51.794300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44169 (local address 127.4.82.130:55969)
0809 00:59:51.795141 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:51.795192 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:51.797014 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:51.797661 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:51.797668 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:51.798154 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:51.798656 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:51.798672 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:51.800179 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:51.800187 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:51.802030 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:51.802046 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:51.803183 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:51.803193 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:51.803551 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:51.804310 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:51.804330 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:51.806397 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:51.809371 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:51.809379 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:51.809393 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:51.809804 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:51.810181 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:51.810186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:51.810190 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:51.810379 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:51.810906 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:51.810914 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:51.811222 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 00:59:51.811536 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 00:59:51.812031 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":388,"threads_started":1}
I20260809 00:59:51.814657 10826 heartbeater.cc:347] Connected to a master server at 127.4.82.190:44169
I20260809 00:59:51.814946 10826 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:51.815685 10826 heartbeater.cc:511] Master 127.4.82.190:44169 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
I20260809 00:59:51.817056 10471 ts_manager.cc:194] Registered new tserver with Master: 8010995d16e944ae8fe3366f75f0702f (127.4.82.130:33203)
I20260809 00:59:51.817862 10471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:55969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 00:59:51.843878 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44169
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:38777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:52.167533 10832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:52.167800 10832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:52.167896 10832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:52.175016 10832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:52.175115 10832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:52.175231 10832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 00:59:52.184998 10832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:38777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:52.186105 10832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:52.187336 10832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:52.199805 10837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:52.199899 10838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:52.200137 10840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:52.200932 10832 server_base.cc:1034] running on GCE node
I20260809 00:59:52.201490 10832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:52.202386 10832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:52.203579 10832 hybrid_clock.cc:648] HybridClock initialized: now 1786237192203547 us; error 29 us; skew 500 ppm
I20260809 00:59:52.203892 10832 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:52.207084 10832 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 00:59:52.208559 10832 webserver.cc:460] Webserver started at http://127.4.82.131:43287/ using document root <none> and password file <none>
I20260809 00:59:52.209226 10832 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:52.209365 10832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:52.209687 10832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:52.212695 10832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "d0d8a0e4e57146a7b69fc88546af25d3"
format_stamp: "Formatted at 2026-08-09 00:59:52 on dist-test-slave-07c2"
server_key: "9a94ec949e852f51afa33fc21c47cae4"
server_key_iv: "789f84a2bb810cb9d875b42bd460e420"
server_key_version: "encryptionkey@0"
I20260809 00:59:52.213495 10832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "d0d8a0e4e57146a7b69fc88546af25d3"
format_stamp: "Formatted at 2026-08-09 00:59:52 on dist-test-slave-07c2"
server_key: "9a94ec949e852f51afa33fc21c47cae4"
server_key_iv: "789f84a2bb810cb9d875b42bd460e420"
server_key_version: "encryptionkey@0"
I20260809 00:59:52.218497 10832 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 00:59:52.221798 10847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:52.222976 10832 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 00:59:52.223156 10832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "d0d8a0e4e57146a7b69fc88546af25d3"
format_stamp: "Formatted at 2026-08-09 00:59:52 on dist-test-slave-07c2"
server_key: "9a94ec949e852f51afa33fc21c47cae4"
server_key_iv: "789f84a2bb810cb9d875b42bd460e420"
server_key_version: "encryptionkey@0"
I20260809 00:59:52.223413 10832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:52.240057 10832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:52.245501 10832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:52.245832 10832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:52.246925 10832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:52.248721 10832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:52.248833 10832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:52.248971 10832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:52.249047 10832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:52.282857 10832 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:41601
I20260809 00:59:52.282900 10967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:41601 every 8 connection(s)
I20260809 00:59:52.284216 10832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 00:59:52.288960 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 10832
I20260809 00:59:52.289152 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 00:59:52.289595 4426 external_mini_cluster.cc:1428] Setting key b0bec6beb4af057b858915e8366de0ce
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 00:59:52.303956 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:52.286672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:48951 (local address 127.4.82.190:44169)
0809 00:59:52.286857 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:52.286862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:52.287988 (+ 1126us) server_negotiation.cc:407] Connection header received
0809 00:59:52.289885 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:52.289892 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:52.289982 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:52.290091 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:52.291556 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.292346 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.293973 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.294408 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.294540 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:52.298023 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:52.298037 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:52.298041 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:52.298072 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:52.300823 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.301517 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.301527 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.301529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.301654 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.302111 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.302115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.302117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.302378 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:52.302540 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:52.303414 (+ 874us) server_negotiation.cc:299] Negotiation successful
0809 00:59:52.303649 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 00:59:52.304982 10970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:52.287089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44169 (local address 127.4.82.131:48951)
0809 00:59:52.287835 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:52.287876 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:52.289589 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:52.290321 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:52.290328 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:52.290815 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:52.291394 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:52.291413 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.292510 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.292515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:52.293830 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:52.293841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.294644 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.294652 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:52.294901 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:52.295550 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:52.295571 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:52.297825 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:52.300957 (+ 3132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:52.300966 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:52.300979 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:52.301388 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:52.301769 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:52.301774 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:52.301777 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:52.301978 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:52.302514 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:52.302522 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:52.302915 (+ 393us) client_negotiation.cc:769] Sending connection context
0809 00:59:52.303213 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 00:59:52.303757 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":322,"threads_started":1}
I20260809 00:59:52.306210 10968 heartbeater.cc:347] Connected to a master server at 127.4.82.190:44169
I20260809 00:59:52.306497 10968 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:52.307193 10968 heartbeater.cc:511] Master 127.4.82.190:44169 requested a full tablet report, sending...
I20260809 00:59:52.308557 10471 ts_manager.cc:194] Registered new tserver with Master: d0d8a0e4e57146a7b69fc88546af25d3 (127.4.82.131:41601)
I20260809 00:59:52.309335 10471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:48951
I20260809 00:59:52.316591 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:59:52.330230 10684 heartbeater.cc:503] Master 127.4.82.190:44169 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/test-admin.keytab.
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:52.346323 4426 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 00:59:52.351521 10975 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:59:52.348892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34632 (local address 127.4.82.129:32845)
0809 00:59:52.349558 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:52.349565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:52.349592 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:59:52.349767 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:52.349774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:52.350022 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:52.350147 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:52.351030 (+ 883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34632: BlockingRecv error: recv got EOF from 127.0.0.1:34632 (error 108)
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":369,"threads_started":1}
W20260809 00:59:52.351516 4426 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/krb5cc' of type FILE
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:52.353032 4426 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:52.354342 4426 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 00:59:52.369055 10975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:52.355799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34642 (local address 127.4.82.129:32845)
0809 00:59:52.355991 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:52.355999 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:52.356241 (+ 242us) server_negotiation.cc:407] Connection header received
0809 00:59:52.356373 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:52.356376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:52.356469 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:52.356545 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:52.357564 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.358488 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.359706 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.360226 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.360317 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:52.363724 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:52.363734 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:52.363747 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:52.363775 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:52.366616 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.367173 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.367179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.367181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.367358 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.367671 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.367674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.367676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.368004 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:52.368250 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:52.368476 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 00:59:52.368769 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 00:59:52.374349 10975 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:59:52.372919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34650 (local address 127.4.82.129:32845)
0809 00:59:52.373116 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:52.373121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:52.373191 (+ 70us) server_negotiation.cc:407] Connection header received
0809 00:59:52.373320 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:52.373323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:52.373371 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:52.373443 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:52.374093 (+ 650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34650: BlockingRecv error: recv got EOF from 127.0.0.1:34650 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 00:59:52.374660 4426 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/krb5cc' of type FILE
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:52.376097 4426 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:52.377381 4426 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 00:59:52.391014 10975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:52.378586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.4.82.129:32845)
0809 00:59:52.378784 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:52.378790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:52.379033 (+ 243us) server_negotiation.cc:407] Connection header received
0809 00:59:52.379226 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:52.379235 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:52.379327 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:52.379433 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:52.380366 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.381036 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.382245 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.382688 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.382814 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:52.386149 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:52.386154 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:52.386157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:52.386198 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:52.388593 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.389222 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.389229 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.389232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.389360 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.389766 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.389771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.389774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.390022 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:52.390292 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:52.390572 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 00:59:52.390774 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 00:59:52.395407 10975 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:59:52.393662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.4.82.129:32845)
0809 00:59:52.393822 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:52.393828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:52.394222 (+ 394us) server_negotiation.cc:407] Connection header received
0809 00:59:52.394368 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:52.394372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:52.394454 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:52.394542 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:52.395149 (+ 607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34672: BlockingRecv error: recv got EOF from 127.0.0.1:34672 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 00:59:52.395637 4426 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786237144174446-4426-0/krb5kdc/krb5cc' of type FILE
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:52.396991 4426 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:52.398238 4426 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 00:59:52.411629 10975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:52.399334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34678 (local address 127.4.82.129:32845)
0809 00:59:52.399516 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:52.399523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:52.399757 (+ 234us) server_negotiation.cc:407] Connection header received
0809 00:59:52.399921 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:52.399926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:52.400007 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:52.400104 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:52.401016 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.401703 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.402944 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.403462 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.403594 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:52.406738 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:52.406744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:52.406749 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:52.406788 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:52.409105 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.409760 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.409778 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.409781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.409935 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.410294 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.410299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.410302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.410542 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:52.410865 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:52.411120 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 00:59:52.411375 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 00:59:52.413115 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 10539
I20260809 00:59:52.424796 10975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:52.413784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34680 (local address 127.4.82.129:32845)
0809 00:59:52.413910 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:52.413914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:52.414339 (+ 425us) server_negotiation.cc:407] Connection header received
0809 00:59:52.414519 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:52.414524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:52.414627 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:52.414718 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:52.415723 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.416372 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.417570 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.417985 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.418120 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:52.420170 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:52.420176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:52.420180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:52.420214 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:52.422352 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.422951 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.422956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.422960 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.423084 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.423512 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.423517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.423520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.423763 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:52.424008 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:52.424322 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 00:59:52.424509 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:59:52.425287 10679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:52.509867 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 10539
I20260809 00:59:52.528406 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 10690
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 00:59:52.544291 10988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:52.529393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58236 (local address 127.4.82.130:33203)
0809 00:59:52.530008 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:52.530015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:52.530040 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:59:52.530111 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:52.530118 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:52.530333 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:52.530467 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:52.531461 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.532497 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.533703 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.534259 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.534361 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:52.538260 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:52.538272 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:52.538288 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:52.538322 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:52.541473 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.542138 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.542146 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.542148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.542291 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.542630 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.542634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.542636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.543052 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:52.543358 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:52.543707 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 00:59:52.543960 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":343,"threads_started":1}
I20260809 00:59:52.545612 10821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:52.627712 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 10690
I20260809 00:59:52.645464 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 10832
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 00:59:52.661583 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:52.646583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47528 (local address 127.4.82.131:41601)
0809 00:59:52.647243 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:52.647250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:52.647279 (+ 29us) server_negotiation.cc:407] Connection header received
0809 00:59:52.647410 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:52.647416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:52.647621 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:52.647752 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:52.648734 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.649885 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.651066 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:52.651742 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:52.651841 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:52.655787 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:52.655798 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:52.655811 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:52.655842 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:52.658744 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.659481 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.659489 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.659491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.659629 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:52.660001 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:52.660005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:52.660007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:52.660374 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:52.660631 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:52.660919 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 00:59:52.661251 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":342,"threads_started":1}
I20260809 00:59:52.662927 10963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:52.749069 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 10832
I20260809 00:59:52.767419 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 10439
I20260809 00:59:52.768590 10503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:52.871003 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 10439
2026-08-09T00:59:52Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3928 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10995](info): set up 2 sockets
Aug 09 00:59:52 dist-test-slave-07c2 krb5kdc[10995](info): commencing operation
krb5kdc: starting...
W20260809 00:59:54.002765 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 00:59:54 dist-test-slave-07c2 krb5kdc[10995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:59:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:59:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.190.keytab.
Entry for principal oryx/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.190.keytab.
I20260809 00:59:54.216621 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:44177
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.190.keytab
--principal=oryx/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:44177
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:54.529438 11009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:54.529704 11009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:54.529794 11009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:54.536727 11009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:54.536826 11009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:54.536872 11009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:54.536931 11009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:54.536975 11009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:54.546272 11009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:44177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.190.keytab
--principal=oryx/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:44177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:54.547510 11009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:54.548740 11009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:54.560072 11014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:54.560024 11016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:54.560189 11018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:54.561220 11009 server_base.cc:1034] running on GCE node
I20260809 00:59:54.561944 11009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:54.563230 11009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:54.564484 11009 hybrid_clock.cc:648] HybridClock initialized: now 1786237194564437 us; error 35 us; skew 500 ppm
I20260809 00:59:54.564821 11009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:54 dist-test-slave-07c2 krb5kdc[10995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237194, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:54.568063 11009 init.cc:377] Logged in from keytab as oryx/127.4.82.190@KRBTEST.COM (short username oryx)
I20260809 00:59:54.569504 11009 webserver.cc:460] Webserver started at http://127.4.82.190:40509/ using document root <none> and password file <none>
I20260809 00:59:54.570170 11009 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:54.570315 11009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:54.570611 11009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:54.573598 11009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "c0bb0e225e8e46a8b608e6094288b96b"
format_stamp: "Formatted at 2026-08-09 00:59:54 on dist-test-slave-07c2"
server_key: "58c04b5412d6d68914f745d432ba6bc5"
server_key_iv: "cccaffee9f639b43822e4b2c15982518"
server_key_version: "encryptionkey@0"
I20260809 00:59:54.574374 11009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "c0bb0e225e8e46a8b608e6094288b96b"
format_stamp: "Formatted at 2026-08-09 00:59:54 on dist-test-slave-07c2"
server_key: "58c04b5412d6d68914f745d432ba6bc5"
server_key_iv: "cccaffee9f639b43822e4b2c15982518"
server_key_version: "encryptionkey@0"
I20260809 00:59:54.579746 11009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260809 00:59:54.582890 11025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:54.584141 11009 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:59:54.584312 11009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "c0bb0e225e8e46a8b608e6094288b96b"
format_stamp: "Formatted at 2026-08-09 00:59:54 on dist-test-slave-07c2"
server_key: "58c04b5412d6d68914f745d432ba6bc5"
server_key_iv: "cccaffee9f639b43822e4b2c15982518"
server_key_version: "encryptionkey@0"
I20260809 00:59:54.584496 11009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:54.629336 11009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:54.633082 11009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:54.633364 11009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:54.650966 11009 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:44177
I20260809 00:59:54.651000 11079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:44177 every 8 connection(s)
I20260809 00:59:54.652272 11009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 00:59:54.655764 11080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:54.657347 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 11009
I20260809 00:59:54.657559 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 00:59:54.658219 4426 external_mini_cluster.cc:1428] Setting key 72ea617e38fcfca33edd6ffe189041ef
I20260809 00:59:54.668566 11080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b: Bootstrap starting.
I20260809 00:59:54.672119 11080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:54.673086 11080 log.cc:824] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b: Log is configured to *not* fsync() on all Append() calls
Aug 09 00:59:54 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 00:59:54.675926 11080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b: No bootstrap required, opened a new log
I20260809 00:59:54.682482 11083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:54.660259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33582 (local address 127.4.82.190:44177)
0809 00:59:54.661071 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:54.661087 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:54.661136 (+ 49us) server_negotiation.cc:407] Connection header received
0809 00:59:54.663564 (+ 2428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:54.663590 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:54.664049 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:54.664471 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:54.665791 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:54.667829 (+ 2038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:54.669097 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:54.670128 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:54.670354 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:54.674434 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:54.674442 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:54.674454 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:54.674486 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:54.678161 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:54.678777 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:54.678785 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:54.678793 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:54.678999 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:54.679410 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:54.679416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:54.679421 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:54.680085 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:54.680423 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:54.680682 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 00:59:54.681077 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":332,"threads_started":1}
I20260809 00:59:54.685982 11080 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } }
I20260809 00:59:54.686296 11080 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:54.686368 11080 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0bb0e225e8e46a8b608e6094288b96b, State: Initialized, Role: FOLLOWER
I20260809 00:59:54.686895 11080 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } }
I20260809 00:59:54.687085 11080 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:59:54.687179 11080 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:59:54.687345 11080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:54.689599 11080 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } }
I20260809 00:59:54.690012 11080 leader_election.cc:302] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0bb0e225e8e46a8b608e6094288b96b; no voters:
I20260809 00:59:54.690521 11080 leader_election.cc:288] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:59:54.690693 11085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:54.691924 11085 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [term 1 LEADER]: Becoming Leader. State: Replica: c0bb0e225e8e46a8b608e6094288b96b, State: Running, Role: LEADER
I20260809 00:59:54.692462 11085 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } }
I20260809 00:59:54.692664 11080 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:54.695492 11086 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } }
I20260809 00:59:54.695586 11087 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0bb0e225e8e46a8b608e6094288b96b. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } }
I20260809 00:59:54.695865 11086 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:54.695865 11087 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:54.696735 11098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:59:54.702050 11098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:59:54.712453 11098 catalog_manager.cc:1347] Generated new cluster ID: 91c290b0219847f5a904518c3570ed7b
I20260809 00:59:54.712565 11098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:59:54.735724 11098 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:59:54.736905 11098 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:59:54.744158 11098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b: Generated new TSK 0
I20260809 00:59:54.744926 11098 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab.
Entry for principal oryx/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab.
I20260809 00:59:54.795087 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44177
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab
--principal=oryx/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:55.114542 11110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:55.114801 11110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:55.114997 11110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:55.122023 11110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:55.122119 11110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:55.122330 11110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 00:59:55.132265 11110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab
--principal=oryx/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:55.133430 11110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:55.134701 11110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:55.147675 11118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:55.147763 11115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:55.147733 11116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:55.148514 11110 server_base.cc:1034] running on GCE node
I20260809 00:59:55.149134 11110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:55.149981 11110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:55.151201 11110 hybrid_clock.cc:648] HybridClock initialized: now 1786237195151158 us; error 57 us; skew 500 ppm
I20260809 00:59:55.151522 11110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:55 dist-test-slave-07c2 krb5kdc[10995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237195, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:55.154650 11110 init.cc:377] Logged in from keytab as oryx/127.4.82.129@KRBTEST.COM (short username oryx)
I20260809 00:59:55.156082 11110 webserver.cc:460] Webserver started at http://127.4.82.129:35463/ using document root <none> and password file <none>
I20260809 00:59:55.156726 11110 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:55.156867 11110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:55.157157 11110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:55.159930 11110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "3d14b4ca8b0140d2a29afca36f844a53"
format_stamp: "Formatted at 2026-08-09 00:59:55 on dist-test-slave-07c2"
server_key: "8d9fcc7b24c64cc2400338b8a2db44fd"
server_key_iv: "14b397413b2688e7f0871fd3021bde34"
server_key_version: "encryptionkey@0"
I20260809 00:59:55.160670 11110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "3d14b4ca8b0140d2a29afca36f844a53"
format_stamp: "Formatted at 2026-08-09 00:59:55 on dist-test-slave-07c2"
server_key: "8d9fcc7b24c64cc2400338b8a2db44fd"
server_key_iv: "14b397413b2688e7f0871fd3021bde34"
server_key_version: "encryptionkey@0"
I20260809 00:59:55.165433 11110 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 00:59:55.168561 11125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:55.169696 11110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:59:55.169878 11110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "3d14b4ca8b0140d2a29afca36f844a53"
format_stamp: "Formatted at 2026-08-09 00:59:55 on dist-test-slave-07c2"
server_key: "8d9fcc7b24c64cc2400338b8a2db44fd"
server_key_iv: "14b397413b2688e7f0871fd3021bde34"
server_key_version: "encryptionkey@0"
I20260809 00:59:55.170076 11110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:55.191529 11110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:55.195549 11110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:55.195779 11110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:55.196573 11110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:55.197767 11110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:55.197837 11110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:55.197916 11110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:55.197978 11110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:55.229453 11110 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:39777
I20260809 00:59:55.229504 11242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:39777 every 8 connection(s)
I20260809 00:59:55.230717 11110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 00:59:55.235460 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 11110
I20260809 00:59:55.235666 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 00:59:55.236301 4426 external_mini_cluster.cc:1428] Setting key a7b5e6510eec66e86a29129288f16ed7
Aug 09 00:59:55 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237195, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 00:59:55.251994 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:55.233278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:41303 (local address 127.4.82.190:44177)
0809 00:59:55.233948 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:55.233953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:55.234740 (+ 787us) server_negotiation.cc:407] Connection header received
0809 00:59:55.236678 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:55.236683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:55.236770 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:55.236899 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:55.238523 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:55.239698 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:55.241165 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:55.241863 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:55.242006 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:55.245638 (+ 3632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:55.245646 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:55.245653 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:55.245705 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:55.248765 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:55.249676 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:55.249680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:55.249683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:55.249812 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:55.250284 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:55.250287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:55.250302 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:55.250555 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:55.250674 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:55.251569 (+ 895us) server_negotiation.cc:299] Negotiation successful
0809 00:59:55.251757 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":228,"threads_started":1}
I20260809 00:59:55.253468 11248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:55.233825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.129:41303)
0809 00:59:55.234573 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:55.234614 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:55.236365 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:55.237186 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:55.237194 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:55.237681 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:55.238354 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:55.238374 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:55.239857 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:55.239861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:55.241018 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:55.241030 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:55.242084 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:55.242089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:55.242272 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:55.242952 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:55.242975 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:55.245425 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:55.248901 (+ 3476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:55.248913 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:55.248930 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:55.249549 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:55.249932 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:55.249939 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:55.249946 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:55.250176 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:55.250702 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:55.250721 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:55.251156 (+ 435us) client_negotiation.cc:769] Sending connection context
0809 00:59:55.251489 (+ 333us) client_negotiation.cc:240] Negotiation successful
0809 00:59:55.252028 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":280,"threads_started":1}
I20260809 00:59:55.254959 11243 heartbeater.cc:347] Connected to a master server at 127.4.82.190:44177
I20260809 00:59:55.255268 11243 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:55.256022 11243 heartbeater.cc:511] Master 127.4.82.190:44177 requested a full tablet report, sending...
I20260809 00:59:55.257964 11043 ts_manager.cc:194] Registered new tserver with Master: 3d14b4ca8b0140d2a29afca36f844a53 (127.4.82.129:39777)
I20260809 00:59:55.259456 11043 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:41303
WARNING: no policy specified for oryx/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab.
Entry for principal oryx/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab.
I20260809 00:59:55.288374 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44177
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab
--principal=oryx/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:55.617635 11251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:55.617923 11251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:55.618083 11251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:55.625111 11251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:55.625283 11251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:55.625456 11251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 00:59:55.634716 11251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab
--principal=oryx/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:55.635957 11251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:55.637231 11251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:55.649772 11256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:55.649750 11257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:55.649806 11259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:55.650590 11251 server_base.cc:1034] running on GCE node
I20260809 00:59:55.651314 11251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:55.652352 11251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:55.653599 11251 hybrid_clock.cc:648] HybridClock initialized: now 1786237195653546 us; error 57 us; skew 500 ppm
I20260809 00:59:55.653925 11251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:55 dist-test-slave-07c2 krb5kdc[10995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237195, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:55.657606 11251 init.cc:377] Logged in from keytab as oryx/127.4.82.130@KRBTEST.COM (short username oryx)
I20260809 00:59:55.664363 11251 webserver.cc:460] Webserver started at http://127.4.82.130:39793/ using document root <none> and password file <none>
I20260809 00:59:55.665038 11251 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:55.665167 11251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:55.665482 11251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:55.668433 11251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "27c702680b924d28995faa26a770ed54"
format_stamp: "Formatted at 2026-08-09 00:59:55 on dist-test-slave-07c2"
server_key: "8d7104dcb8a4e5f694ba43ff5dba8672"
server_key_iv: "7154fd896f07875571a0e1c93c244fe6"
server_key_version: "encryptionkey@0"
I20260809 00:59:55.669176 11251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "27c702680b924d28995faa26a770ed54"
format_stamp: "Formatted at 2026-08-09 00:59:55 on dist-test-slave-07c2"
server_key: "8d7104dcb8a4e5f694ba43ff5dba8672"
server_key_iv: "7154fd896f07875571a0e1c93c244fe6"
server_key_version: "encryptionkey@0"
I20260809 00:59:55.674160 11251 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 00:59:55.677408 11266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:55.678622 11251 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 00:59:55.678798 11251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "27c702680b924d28995faa26a770ed54"
format_stamp: "Formatted at 2026-08-09 00:59:55 on dist-test-slave-07c2"
server_key: "8d7104dcb8a4e5f694ba43ff5dba8672"
server_key_iv: "7154fd896f07875571a0e1c93c244fe6"
server_key_version: "encryptionkey@0"
I20260809 00:59:55.679019 11251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:55.713058 11251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:55.717207 11251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:55.717489 11251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:55.718452 11251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:55.719790 11251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:55.719892 11251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:55.720050 11251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:55.720125 11251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:55.753715 11251 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:42733
I20260809 00:59:55.753803 11385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:42733 every 8 connection(s)
I20260809 00:59:55.755148 11251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 00:59:55.758895 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 11251
I20260809 00:59:55.759195 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 00:59:55.759860 4426 external_mini_cluster.cc:1428] Setting key a75b2ef6928ecfdcbe9069d57790ac58
Aug 09 00:59:55 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237195, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 00:59:55.775799 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:55.757816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:41601 (local address 127.4.82.190:44177)
0809 00:59:55.758234 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:55.758239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:55.759478 (+ 1239us) server_negotiation.cc:407] Connection header received
0809 00:59:55.761487 (+ 2009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:55.761493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:55.761589 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:55.761715 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:55.763144 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:55.763942 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:55.765446 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:55.765839 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:55.765971 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:55.769416 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:55.769421 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:55.769424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:55.769454 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:55.772523 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:55.773276 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:55.773283 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:55.773290 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:55.773450 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:55.773888 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:55.773894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:55.773898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:55.774150 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:55.774360 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:55.775156 (+ 796us) server_negotiation.cc:299] Negotiation successful
0809 00:59:55.775415 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":211,"threads_started":1}
I20260809 00:59:55.776719 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:55.758417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.130:41601)
0809 00:59:55.759256 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:55.759354 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:55.761231 (+ 1877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:55.761946 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:55.761955 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:55.762446 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:55.762975 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:55.762991 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:55.764103 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:55.764108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:55.765306 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:55.765317 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:55.766061 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:55.766067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:55.766294 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:55.766991 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:55.767018 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:55.769203 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:55.772663 (+ 3460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:55.772673 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:55.772687 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:55.773135 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:55.773584 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:55.773588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:55.773592 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:55.773773 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:55.774426 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:55.774432 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:55.774798 (+ 366us) client_negotiation.cc:769] Sending connection context
0809 00:59:55.775093 (+ 295us) client_negotiation.cc:240] Negotiation successful
0809 00:59:55.775635 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":341,"threads_started":1}
I20260809 00:59:55.778098 11387 heartbeater.cc:347] Connected to a master server at 127.4.82.190:44177
I20260809 00:59:55.778456 11387 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:55.779348 11387 heartbeater.cc:511] Master 127.4.82.190:44177 requested a full tablet report, sending...
I20260809 00:59:55.781023 11043 ts_manager.cc:194] Registered new tserver with Master: 27c702680b924d28995faa26a770ed54 (127.4.82.130:42733)
I20260809 00:59:55.781854 11043 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:41601
WARNING: no policy specified for oryx/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab.
Entry for principal oryx/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab.
I20260809 00:59:55.815100 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44177
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab
--principal=oryx/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:56.134621 11394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:56.134873 11394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:56.134964 11394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:56.141780 11394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:56.141875 11394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:56.141975 11394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 00:59:56.151060 11394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab
--principal=oryx/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:44177
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:56.152163 11394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:56.153316 11394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:56.165926 11401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:56.166153 11403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:56.166332 11400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:56.167181 11394 server_base.cc:1034] running on GCE node
I20260809 00:59:56.167806 11394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:56.168707 11394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:56.169936 11394 hybrid_clock.cc:648] HybridClock initialized: now 1786237196169880 us; error 46 us; skew 500 ppm
I20260809 00:59:56.170243 11394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:56 dist-test-slave-07c2 krb5kdc[10995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237196, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:56.173385 11394 init.cc:377] Logged in from keytab as oryx/127.4.82.131@KRBTEST.COM (short username oryx)
I20260809 00:59:56.174860 11394 webserver.cc:460] Webserver started at http://127.4.82.131:33957/ using document root <none> and password file <none>
I20260809 00:59:56.175601 11394 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:56.175753 11394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:56.176044 11394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:56.179273 11394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "964f6e9dde52455299287e03815ae164"
format_stamp: "Formatted at 2026-08-09 00:59:56 on dist-test-slave-07c2"
server_key: "082a6b99e0d05d41870fb4b7d8ebe191"
server_key_iv: "9481e1631628df53af6668bfe2fd09fb"
server_key_version: "encryptionkey@0"
I20260809 00:59:56.180233 11394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "964f6e9dde52455299287e03815ae164"
format_stamp: "Formatted at 2026-08-09 00:59:56 on dist-test-slave-07c2"
server_key: "082a6b99e0d05d41870fb4b7d8ebe191"
server_key_iv: "9481e1631628df53af6668bfe2fd09fb"
server_key_version: "encryptionkey@0"
I20260809 00:59:56.185336 11394 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 00:59:56.188670 11410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:56.189921 11394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 00:59:56.190093 11394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "964f6e9dde52455299287e03815ae164"
format_stamp: "Formatted at 2026-08-09 00:59:56 on dist-test-slave-07c2"
server_key: "082a6b99e0d05d41870fb4b7d8ebe191"
server_key_iv: "9481e1631628df53af6668bfe2fd09fb"
server_key_version: "encryptionkey@0"
I20260809 00:59:56.190296 11394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:56.205261 11394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:56.208916 11394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:56.209197 11394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:56.210086 11394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:56.211335 11394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:56.211433 11394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:56.211602 11394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:56.211690 11394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:56.245908 11394 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:39315
I20260809 00:59:56.245991 11528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:39315 every 8 connection(s)
I20260809 00:59:56.247187 11394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 00:59:56.257316 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 11394
I20260809 00:59:56.257529 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 00:59:56.258054 4426 external_mini_cluster.cc:1428] Setting key 220041b3cafa776bad259e9df2c1cbbb
Aug 09 00:59:56 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237196, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 00:59:56.262682 11243 heartbeater.cc:503] Master 127.4.82.190:44177 was elected leader, sending a full tablet report...
I20260809 00:59:56.267023 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.249553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:36127 (local address 127.4.82.190:44177)
0809 00:59:56.249731 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:56.249736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:56.250756 (+ 1020us) server_negotiation.cc:407] Connection header received
0809 00:59:56.252598 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:56.252605 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:56.252697 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:56.252807 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:56.254210 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.254961 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.256538 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.257122 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.257291 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:56.261118 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:56.261124 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:56.261128 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:56.261173 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:56.264106 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.264800 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.264808 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.264816 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.264979 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.265409 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.265414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.265417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.265701 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:56.265869 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:56.266493 (+ 624us) server_negotiation.cc:299] Negotiation successful
0809 00:59:56.266702 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 00:59:56.267915 11531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.249906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.131:36127)
0809 00:59:56.250586 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:56.250620 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:56.252328 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:56.253035 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:56.253042 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:56.253510 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:56.254031 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:56.254047 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.255122 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.255126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:56.256393 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:56.256405 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.257350 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.257354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:56.257644 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:56.258364 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:56.258397 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:56.260906 (+ 2509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:56.264236 (+ 3330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:56.264244 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:56.264257 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:56.264668 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:56.265082 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:56.265087 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:56.265091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:56.265289 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:56.265837 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:56.265844 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:56.266143 (+ 299us) client_negotiation.cc:769] Sending connection context
0809 00:59:56.266404 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 00:59:56.266853 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":269,"threads_started":1}
I20260809 00:59:56.269119 11529 heartbeater.cc:347] Connected to a master server at 127.4.82.190:44177
I20260809 00:59:56.269411 11529 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:56.270083 11529 heartbeater.cc:511] Master 127.4.82.190:44177 requested a full tablet report, sending...
I20260809 00:59:56.271404 11043 ts_manager.cc:194] Registered new tserver with Master: 964f6e9dde52455299287e03815ae164 (127.4.82.131:39315)
I20260809 00:59:56.272466 11043 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:36127
I20260809 00:59:56.273353 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.82.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.82.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.82.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.189.keytab.
Entry for principal oryx/127.4.82.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.189.keytab.
I20260809 00:59:56.324812 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.189:33311
--webserver_interface=127.4.82.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.189.keytab
--principal=oryx/127.4.82.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:44177,127.4.82.189:33311
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:56.642525 11534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:56.642791 11534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:56.642876 11534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:56.649797 11534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:56.649889 11534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:56.649935 11534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:56.649992 11534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:56.650029 11534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:56.664121 11534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:44177,127.4.82.189:33311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.189.keytab
--principal=oryx/127.4.82.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.189:33311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.82.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
W20260809 00:59:56.664427 11534 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.4.82.190:44177,127.4.82.189:33311'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 00:59:56.665438 11534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:56.666651 11534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:56.678244 11543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:56.678323 11540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:56.678290 11541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:56.679404 11534 server_base.cc:1034] running on GCE node
I20260809 00:59:56.680109 11534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:56.681375 11534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:56.682612 11534 hybrid_clock.cc:648] HybridClock initialized: now 1786237196682559 us; error 52 us; skew 500 ppm
I20260809 00:59:56.682930 11534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:56 dist-test-slave-07c2 krb5kdc[10995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237196, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:56.685962 11534 init.cc:377] Logged in from keytab as oryx/127.4.82.189@KRBTEST.COM (short username oryx)
I20260809 00:59:56.687408 11534 webserver.cc:460] Webserver started at http://127.4.82.189:42127/ using document root <none> and password file <none>
I20260809 00:59:56.688112 11534 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:56.688236 11534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:56.688627 11534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:56.691627 11534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/data/instance:
uuid: "1afdefeb53264bc984c30489b073cfde"
format_stamp: "Formatted at 2026-08-09 00:59:56 on dist-test-slave-07c2"
server_key: "da3d0284c8ea66fb3b5d7b40b76310be"
server_key_iv: "7feeeb6c091fc1b80df2140476f12ecc"
server_key_version: "encryptionkey@0"
I20260809 00:59:56.692440 11534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/wal/instance:
uuid: "1afdefeb53264bc984c30489b073cfde"
format_stamp: "Formatted at 2026-08-09 00:59:56 on dist-test-slave-07c2"
server_key: "da3d0284c8ea66fb3b5d7b40b76310be"
server_key_iv: "7feeeb6c091fc1b80df2140476f12ecc"
server_key_version: "encryptionkey@0"
I20260809 00:59:56.697669 11534 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 00:59:56.701279 11552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:56.702476 11534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 00:59:56.702659 11534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/wal
uuid: "1afdefeb53264bc984c30489b073cfde"
format_stamp: "Formatted at 2026-08-09 00:59:56 on dist-test-slave-07c2"
server_key: "da3d0284c8ea66fb3b5d7b40b76310be"
server_key_iv: "7feeeb6c091fc1b80df2140476f12ecc"
server_key_version: "encryptionkey@0"
I20260809 00:59:56.702879 11534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:56.741264 11534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:56.745079 11534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:56.745347 11534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:56.764124 11534 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.189:33311
I20260809 00:59:56.764153 11606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.189:33311 every 8 connection(s)
I20260809 00:59:56.765388 11534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/data/info.pb
I20260809 00:59:56.768954 11607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:56.774223 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 11534
I20260809 00:59:56.775276 11607 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:59:56.785050 11387 heartbeater.cc:503] Master 127.4.82.190:44177 was elected leader, sending a full tablet report...
Aug 09 00:59:56 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237196, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.189@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
Aug 09 00:59:56 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.82.189@KRBTEST.COM
I20260809 00:59:56.796358 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.778234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33592 (local address 127.4.82.190:44177)
0809 00:59:56.778702 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:56.778719 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:56.779342 (+ 623us) server_negotiation.cc:407] Connection header received
0809 00:59:56.779758 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:56.779763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:56.779850 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:56.779966 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:56.781674 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.782872 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.785014 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.785575 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.785699 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:56.789827 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:56.789832 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:56.789835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:56.789865 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:56.792777 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.793820 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.793826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.793836 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.793968 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.794437 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.794449 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.794451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.794873 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:56.795036 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:56.795762 (+ 726us) server_negotiation.cc:299] Negotiation successful
0809 00:59:56.795984 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":243,"threads_started":1}
I20260809 00:59:56.797293 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.776353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43072 (local address 127.4.82.189:33311)
0809 00:59:56.777082 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:56.777096 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:56.777145 (+ 49us) server_negotiation.cc:407] Connection header received
0809 00:59:56.779337 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:56.779364 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:56.779603 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:56.779815 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:56.781672 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.783380 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.785163 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.785927 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.786116 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:56.790732 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:56.790740 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:56.790751 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:56.790784 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:56.793617 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.794491 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.794497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.794499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.794613 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.795070 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.795075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.795079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.795530 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:56.795821 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:56.796167 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 00:59:56.796715 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":512,"thread_start_us":287,"threads_started":1}
I20260809 00:59:56.797317 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.778235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:33592)
0809 00:59:56.779161 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:56.779198 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:56.779521 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:56.780065 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:56.780074 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:56.780932 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:56.781504 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:56.781527 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.782997 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.783001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:56.784867 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:56.784883 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.785800 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.785808 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:56.786012 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:56.786873 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:56.786893 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:56.789633 (+ 2740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:56.792981 (+ 3348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:56.792992 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:56.793002 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:56.793682 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:56.794081 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:56.794086 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:56.794090 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:56.794316 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:56.795023 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:56.795030 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:56.795374 (+ 344us) client_negotiation.cc:769] Sending connection context
0809 00:59:56.795643 (+ 269us) client_negotiation.cc:240] Negotiation successful
0809 00:59:56.796126 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":53,"thread_start_us":511,"threads_started":1}
I20260809 00:59:56.799175 11607 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:59:56 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237196, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.189@KRBTEST.COM for oryx/127.4.82.189@KRBTEST.COM
I20260809 00:59:56.810973 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.799861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:33311 (local address 127.0.0.1:43076)
0809 00:59:56.800029 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:56.800048 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:56.800143 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:56.800464 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:56.800469 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:56.800787 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:56.800900 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:56.800907 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.801805 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.801808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:56.802628 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:56.802638 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.803258 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.803262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:56.803476 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:56.804028 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:56.804044 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:56.805917 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:56.808668 (+ 2751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:56.808675 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:56.808679 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:56.809048 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:56.809395 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:56.809401 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:56.809406 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:56.809526 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:56.810012 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:56.810017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:56.810147 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 00:59:56.810279 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 00:59:56.810556 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 00:59:56.811012 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.800013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.4.82.189:33311)
0809 00:59:56.800187 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:56.800192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:56.800216 (+ 24us) server_negotiation.cc:407] Connection header received
0809 00:59:56.800259 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:56.800264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:56.800330 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:56.800412 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:56.801015 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.801685 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.802744 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.803131 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.803230 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:56.806091 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:56.806095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:56.806098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:56.806123 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:56.808529 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.809160 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.809167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.809170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.809291 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.809626 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.809634 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.809637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.809883 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:56.810034 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:56.810358 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 00:59:56.810675 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":57}
I20260809 00:59:56.815366 11607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: Bootstrap starting.
I20260809 00:59:56.818259 11607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:56.819231 11607 log.cc:824] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:56.821658 11607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: No bootstrap required, opened a new log
I20260809 00:59:56.828617 11607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } }
I20260809 00:59:56.828938 11607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:56.829032 11607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1afdefeb53264bc984c30489b073cfde, State: Initialized, Role: FOLLOWER
I20260809 00:59:56.829588 11607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } }
I20260809 00:59:56.830381 11614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } } }
I20260809 00:59:56.830703 11614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:59:56.831384 11607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: configured and running, proceeding with master startup.
W20260809 00:59:56.839874 11626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:59:56.840031 11626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:59:56.842293 11534 master_runner.cc:428] Detected that this master 1afdefeb53264bc984c30489b073cfde is joining an existing cluster
I20260809 00:59:56.842375 11534 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.82.189:33311
I20260809 00:59:56.862406 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.845963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.4.82.190:44177)
0809 00:59:56.846171 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:56.846180 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:56.846785 (+ 605us) server_negotiation.cc:407] Connection header received
0809 00:59:56.847100 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:56.847105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:56.847200 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:56.847257 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:56.848640 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.849682 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.851890 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.852562 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.852673 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:56.855411 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:56.855418 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:56.855423 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:56.855462 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:56.859052 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.859940 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.859946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.859950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.860110 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.860588 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.860594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.860597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.860950 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:56.861087 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:56.861781 (+ 694us) server_negotiation.cc:299] Negotiation successful
0809 00:59:56.862028 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 00:59:56.862843 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.846557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43082 (local address 127.4.82.189:33311)
0809 00:59:56.846700 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:56.846705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:56.847359 (+ 654us) server_negotiation.cc:407] Connection header received
0809 00:59:56.847504 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:56.847509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:56.847581 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:56.847692 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:56.849019 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.849994 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.851924 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.852669 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.852770 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:56.855572 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:56.855584 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:56.855591 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:56.855629 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:56.859600 (+ 3971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.860503 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.860511 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.860516 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.860677 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.861085 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.861092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.861098 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.861536 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:56.861679 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:56.862150 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 00:59:56.862418 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47,"spinlock_wait_cycles":896}
I20260809 00:59:56.863932 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.845950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:33596)
0809 00:59:56.846671 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:56.846694 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:56.846926 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:56.847514 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:56.847518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:56.848167 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:56.848479 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:56.848488 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.849828 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.849833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:56.851718 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:56.851733 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.852963 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.852970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:56.853170 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:56.853836 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:56.853862 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:56.855151 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:56.859222 (+ 4071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:56.859228 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:56.859232 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:56.859793 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:56.860275 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:56.860279 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:56.860283 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:56.860448 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:56.861175 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:56.861180 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:56.861405 (+ 225us) client_negotiation.cc:769] Sending connection context
0809 00:59:56.863051 (+ 1646us) client_negotiation.cc:240] Negotiation successful
0809 00:59:56.863421 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":4,"thread_start_us":590,"threads_started":1}
I20260809 00:59:56.863965 11633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.846046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:33311 (local address 127.0.0.1:43082)
0809 00:59:56.847236 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:56.847254 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:56.847390 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:56.847742 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:56.847748 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:56.848510 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:56.848866 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:56.848880 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.850185 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.850190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:56.851779 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:56.851796 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.853098 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.853105 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:56.853337 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:56.854054 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:56.854082 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:56.855410 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:56.859737 (+ 4327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:56.859744 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:56.859750 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:56.860388 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:56.860805 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:56.860812 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:56.860818 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:56.860977 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:56.861745 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:56.861753 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:56.861904 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 00:59:56.863213 (+ 1309us) client_negotiation.cc:240] Negotiation successful
0809 00:59:56.863630 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1060,"mutex_wait_us":63,"spinlock_wait_cycles":21376,"thread_start_us":865,"threads_started":1}
WARNING: no policy specified for oryx/127.4.82.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.82.188@KRBTEST.COM" created.
Aug 09 00:59:56 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237194, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.190@KRBTEST.COM for oryx/127.4.82.189@KRBTEST.COM
I20260809 00:59:56.889349 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.870648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:33311 (local address 127.0.0.1:43086)
0809 00:59:56.871362 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:56.871410 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:56.871623 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:56.873103 (+ 1480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:56.873111 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:56.873968 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:56.874352 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:56.874365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.875926 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.875932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:56.877494 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:56.877510 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.878655 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.878670 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:56.878883 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:56.879619 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:56.879644 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:56.883273 (+ 3629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:56.886376 (+ 3103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:56.886390 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:56.886398 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:56.886795 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:56.887226 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:56.887231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:56.887235 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:56.887470 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:56.888046 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:56.888055 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:56.888271 (+ 216us) client_negotiation.cc:769] Sending connection context
0809 00:59:56.888466 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 00:59:56.888853 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":293,"threads_started":1}
I20260809 00:59:56.889691 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.870654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43086 (local address 127.4.82.189:33311)
0809 00:59:56.870825 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:56.870831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:56.871493 (+ 662us) server_negotiation.cc:407] Connection header received
0809 00:59:56.872751 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:56.872764 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:56.872934 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:56.873035 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:56.874524 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.875776 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.877658 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:56.878473 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:56.878616 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:56.883538 (+ 4922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:56.883543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:56.883546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:56.883574 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:56.886241 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.886921 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.886942 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.886948 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.887106 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:56.887580 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:56.887587 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:56.887590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:56.887929 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:56.888074 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:56.888579 (+ 505us) server_negotiation.cc:299] Negotiation successful
0809 00:59:56.889132 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 00:59:56.891893 11042 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.82.189:33311: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1afdefeb53264bc984c30489b073cfde"
member_type: NON_VOTER
last_known_addr {
host: "127.4.82.189"
port: 33311
}
attrs {
promote: true
}
}
dest_uuid: "c0bb0e225e8e46a8b608e6094288b96b"
cas_config_opid_index: -1
I20260809 00:59:56.893051 11042 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: NON_VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: true } }
I20260809 00:59:56.904012 11581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:56.906723 11638 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1afdefeb53264bc984c30489b073cfde (127.4.82.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: NON_VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: true } } }
I20260809 00:59:56.908051 11581 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 FOLLOWER]: Refusing update from remote peer c0bb0e225e8e46a8b608e6094288b96b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 00:59:56.909096 11639 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: NON_VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:56.912433 11638 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.82.189:33311
I20260809 00:59:56.913952 11639 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: NON_VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: true } } }
I20260809 00:59:56.914163 11534 master.cc:537] Master@127.4.82.189:33311 shutting down...
I20260809 00:59:56.914242 11639 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.82.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.188.keytab.
Entry for principal oryx/127.4.82.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.188.keytab.
I20260809 00:59:56.925242 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.188:42363
--webserver_interface=127.4.82.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.188.keytab
--principal=oryx/127.4.82.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:44177,127.4.82.189:33311,127.4.82.188:42363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:59:56.933656 11581 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1afdefeb53264bc984c30489b073cfde (127.4.82.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: NON_VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: true } } }
I20260809 00:59:56.960646 11614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0bb0e225e8e46a8b608e6094288b96b. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: NON_VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: true } } }
I20260809 00:59:56.960860 11644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: NON_VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: true } } }
I20260809 00:59:56.961061 11644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: This master's current role is: LEARNER
I20260809 00:59:56.960975 11614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: This master's current role is: LEARNER
I20260809 00:59:56.961373 11648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: NON_VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: true } } }
I20260809 00:59:56.961742 11648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: This master's current role is: LEARNER
I20260809 00:59:56.964370 11534 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 LEARNER]: Raft consensus shutting down.
I20260809 00:59:56.964900 11534 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 LEARNER]: Raft consensus is shut down!
I20260809 00:59:56.965088 11534 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: stopping tablet replica
I20260809 00:59:56.973067 11534 master.cc:552] Master@127.4.82.189:33311 shutdown complete.
I20260809 00:59:56.973351 11534 master_runner.cc:305] Clearing existing system tablet
I20260809 00:59:56.973929 11534 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:56.974102 11534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:56.977178 11534 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 00:59:56.978924 11650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:56.979279 11534 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:56.979758 11534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 00:59:56.979889 11534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/wal
uuid: "1afdefeb53264bc984c30489b073cfde"
format_stamp: "Formatted at 2026-08-09 00:59:56 on dist-test-slave-07c2"
server_key: "da3d0284c8ea66fb3b5d7b40b76310be"
server_key_iv: "7feeeb6c091fc1b80df2140476f12ecc"
server_key_version: "encryptionkey@0"
I20260809 00:59:56.984536 11534 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:56.991777 11534 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 00:59:56.992063 11534 log.cc:1196] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 00:59:56.992708 11534 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: Deleting consensus metadata
I20260809 00:59:56.993100 11534 master_runner.cc:315] Copying system tablet from 127.4.82.190:44177
I20260809 00:59:56.996331 11534 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: tablet copy: Beginning tablet copy session from remote peer at address 127.4.82.190:44177
I20260809 00:59:57.012728 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.997492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33598 (local address 127.4.82.190:44177)
0809 00:59:56.997681 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:56.997688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:56.997980 (+ 292us) server_negotiation.cc:407] Connection header received
0809 00:59:56.998298 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:56.998306 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:56.998412 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:56.998489 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:56.999773 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.000903 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.002774 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.003463 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.003613 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.005942 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:57.005949 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:57.005953 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:57.005993 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:57.009659 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.010455 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.010461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.010464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.010636 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.011067 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.011071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.011074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.011465 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:57.011620 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:57.012070 (+ 450us) server_negotiation.cc:299] Negotiation successful
0809 00:59:57.012298 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 00:59:57.012761 11657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:56.997208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:33598)
0809 00:59:56.997874 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:56.997895 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:56.998113 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:56.998734 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:56.998739 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:56.999334 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:56.999594 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:56.999607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.001130 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.001135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.002607 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.002622 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.003641 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.003665 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.003859 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.004603 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:57.004630 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:57.005729 (+ 1099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:57.009811 (+ 4082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.009817 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.009821 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.010329 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.010759 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.010768 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.010773 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.010948 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.011595 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:57.011601 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:57.011773 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 00:59:57.011953 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 00:59:57.012287 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":500,"threads_started":1}
I20260809 00:59:57.013976 11065 tablet_copy_service.cc:140] P c0bb0e225e8e46a8b608e6094288b96b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1afdefeb53264bc984c30489b073cfde ({username='oryx', principal='oryx/127.4.82.189@KRBTEST.COM'} at 127.0.0.1:33598)
I20260809 00:59:57.014235 11065 tablet_copy_service.cc:161] P c0bb0e225e8e46a8b608e6094288b96b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1afdefeb53264bc984c30489b073cfde at {username='oryx', principal='oryx/127.4.82.189@KRBTEST.COM'} at 127.0.0.1:33598: session id = 1afdefeb53264bc984c30489b073cfde-00000000000000000000000000000000
I20260809 00:59:57.018790 11065 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 00:59:57.020896 11534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 00:59:57.028012 11534 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: tablet copy: Starting download of 0 data blocks...
I20260809 00:59:57.028357 11534 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: tablet copy: Starting download of 1 WAL segments...
I20260809 00:59:57.031347 11534 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 00:59:57.035655 11065 tablet_copy_service.cc:342] P c0bb0e225e8e46a8b608e6094288b96b: Request end of tablet copy session 1afdefeb53264bc984c30489b073cfde-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.82.189@KRBTEST.COM'} at 127.0.0.1:33598
I20260809 00:59:57.035864 11065 tablet_copy_service.cc:433] P c0bb0e225e8e46a8b608e6094288b96b: ending tablet copy session 1afdefeb53264bc984c30489b073cfde-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1afdefeb53264bc984c30489b073cfde
I20260809 00:59:57.048242 11534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:57.050907 11658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:57.052522 11659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:57.063032 11661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:57.064443 11534 server_base.cc:1034] running on GCE node
I20260809 00:59:57.064847 11534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:57.065513 11534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:57.066736 11534 hybrid_clock.cc:648] HybridClock initialized: now 1786237197066670 us; error 38 us; skew 500 ppm
I20260809 00:59:57.066958 11534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:57 dist-test-slave-07c2 krb5kdc[10995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237197, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:57.069227 11534 init.cc:377] Logged in from keytab as oryx/127.4.82.189@KRBTEST.COM (short username oryx)
I20260809 00:59:57.072186 11534 webserver.cc:460] Webserver started at http://127.4.82.189:40665/ using document root <none> and password file <none>
I20260809 00:59:57.072685 11534 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:57.073179 11534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:57.076665 11534 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 00:59:57.078989 11668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:57.079483 11534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 00:59:57.079669 11534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/wal
uuid: "1afdefeb53264bc984c30489b073cfde"
format_stamp: "Formatted at 2026-08-09 00:59:56 on dist-test-slave-07c2"
server_key: "da3d0284c8ea66fb3b5d7b40b76310be"
server_key_iv: "7feeeb6c091fc1b80df2140476f12ecc"
server_key_version: "encryptionkey@0"
I20260809 00:59:57.079905 11534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:57.104936 11534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:57.109272 11534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:57.138036 11534 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.189:33311
I20260809 00:59:57.138125 11723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.189:33311 every 8 connection(s)
I20260809 00:59:57.139340 11534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-1/data/info.pb
I20260809 00:59:57.145545 11724 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:59:57.149688 11724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: Bootstrap starting.
I20260809 00:59:57.163188 11724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:59:57.163776 11724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: Bootstrap complete.
I20260809 00:59:57.164805 11724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: NON_VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: true } }
I20260809 00:59:57.165081 11724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1afdefeb53264bc984c30489b073cfde, State: Initialized, Role: LEARNER
I20260809 00:59:57.165393 11724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: NON_VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: true } }
I20260809 00:59:57.166293 11727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: NON_VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: true } } }
I20260809 00:59:57.166577 11724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:57.166711 11727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: This master's current role is: LEARNER
I20260809 00:59:57.175595 11738 catalog_manager.cc:1259] Loaded cluster ID: 91c290b0219847f5a904518c3570ed7b
I20260809 00:59:57.175704 11738 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: loading cluster ID for follower catalog manager: success
I20260809 00:59:57.179450 11738 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: acquiring CA information for follower catalog manager: success
I20260809 00:59:57.181787 11738 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:59:57.274986 11529 heartbeater.cc:503] Master 127.4.82.190:44177 was elected leader, sending a full tablet report...
I20260809 00:59:57.325922 11640 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: NON_VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: true } } }
I20260809 00:59:57.326397 11640 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:57.332324 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.326194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:33311 (local address 127.0.0.1:43098)
0809 00:59:57.326729 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:57.326756 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:57.326903 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:57.327426 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:57.327431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:57.327462 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:57.327686 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.327696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.329056 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.329062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.330330 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.330342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.331499 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.331504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.331711 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.331786 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 00:59:57.331895 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 00:59:57.332038 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381}
I20260809 00:59:57.332423 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.326411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.4.82.189:33311)
0809 00:59:57.327077 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:57.327081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:57.327100 (+ 19us) server_negotiation.cc:407] Connection header received
0809 00:59:57.327164 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:57.327168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:57.327235 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:57.327385 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:57.327879 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.328894 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.330492 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.331323 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.331470 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.331709 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:57.331937 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 00:59:57.332067 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":346,"threads_started":1}
I20260809 00:59:57.333753 11727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0bb0e225e8e46a8b608e6094288b96b. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: NON_VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: true } } }
I20260809 00:59:57.334028 11727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: This master's current role is: LEARNER
I20260809 00:59:57.334551 11640 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b: attempting to promote NON_VOTER 1afdefeb53264bc984c30489b073cfde to VOTER
I20260809 00:59:57.335314 11640 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } }
I20260809 00:59:57.336895 11697 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 LEARNER]: Refusing update from remote peer c0bb0e225e8e46a8b608e6094288b96b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 00:59:57.337420 11639 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 00:59:57.340219 11640 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 1afdefeb53264bc984c30489b073cfde (127.4.82.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } }
W20260809 00:59:57.341742 11647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:57.342074 11647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:57.342238 11647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 00:59:57.342370 11697 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 1afdefeb53264bc984c30489b073cfde (127.4.82.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } }
I20260809 00:59:57.343384 11638 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } }
I20260809 00:59:57.343338 11639 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } }
I20260809 00:59:57.343576 11639 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:57.343593 11638 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:57.346474 11727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } }
I20260809 00:59:57.346728 11727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:59:57.352028 11647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:59:57.352142 11647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:57.352234 11647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:57.352317 11647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:59:57.352381 11647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:59:57.362427 11647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:44177,127.4.82.189:33311,127.4.82.188:42363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.188.keytab
--principal=oryx/127.4.82.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.188:42363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.82.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:57.363723 11647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:57.364955 11647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:57.376735 11750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:57.376771 11748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:57.376920 11747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:57.378165 11647 server_base.cc:1034] running on GCE node
I20260809 00:59:57.378932 11647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:57.380299 11647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:57.381551 11647 hybrid_clock.cc:648] HybridClock initialized: now 1786237197381505 us; error 32 us; skew 500 ppm
I20260809 00:59:57.381912 11647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:57 dist-test-slave-07c2 krb5kdc[10995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237197, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:57.385028 11647 init.cc:377] Logged in from keytab as oryx/127.4.82.188@KRBTEST.COM (short username oryx)
I20260809 00:59:57.386325 11647 webserver.cc:460] Webserver started at http://127.4.82.188:43821/ using document root <none> and password file <none>
I20260809 00:59:57.386970 11647 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:57.387111 11647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:57.387478 11647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:59:57.390519 11647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/data/instance:
uuid: "43b38d7e56b6483885029e8887324a97"
format_stamp: "Formatted at 2026-08-09 00:59:57 on dist-test-slave-07c2"
server_key: "b8256b459693caf18ad69007cfc2c5a2"
server_key_iv: "99aeca865814af8f05b3920cfe5ca03f"
server_key_version: "encryptionkey@0"
I20260809 00:59:57.391340 11647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/wal/instance:
uuid: "43b38d7e56b6483885029e8887324a97"
format_stamp: "Formatted at 2026-08-09 00:59:57 on dist-test-slave-07c2"
server_key: "b8256b459693caf18ad69007cfc2c5a2"
server_key_iv: "99aeca865814af8f05b3920cfe5ca03f"
server_key_version: "encryptionkey@0"
I20260809 00:59:57.397219 11647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 00:59:57.400566 11757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:57.401793 11647 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260809 00:59:57.401957 11647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/wal
uuid: "43b38d7e56b6483885029e8887324a97"
format_stamp: "Formatted at 2026-08-09 00:59:57 on dist-test-slave-07c2"
server_key: "b8256b459693caf18ad69007cfc2c5a2"
server_key_iv: "99aeca865814af8f05b3920cfe5ca03f"
server_key_version: "encryptionkey@0"
I20260809 00:59:57.402130 11647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:57.425397 11647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:57.429164 11647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:57.429385 11647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:57.448082 11647 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.188:42363
I20260809 00:59:57.448133 11809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.188:42363 every 8 connection(s)
I20260809 00:59:57.449735 11647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/data/info.pb
I20260809 00:59:57.453424 11810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:57.456367 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 11647
I20260809 00:59:57.462714 11810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:59:57 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.82.188@KRBTEST.COM
Aug 09 00:59:57 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237197, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.188@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 00:59:57.480173 11813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.458533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47872 (local address 127.4.82.188:42363)
0809 00:59:57.459265 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:57.459276 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:57.459366 (+ 90us) server_negotiation.cc:407] Connection header received
0809 00:59:57.460900 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:57.460924 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:57.461274 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:57.461616 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:57.463103 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.464892 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.466434 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.467473 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.467664 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.472418 (+ 4754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:57.472429 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:57.472439 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:57.472468 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:57.475991 (+ 3523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.476894 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.476900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.476906 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.477045 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.477468 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.477471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.477473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.477924 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:57.478267 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:57.478714 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 00:59:57.479066 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":307,"threads_started":1}
I20260809 00:59:57.480430 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.465638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33600 (local address 127.4.82.190:44177)
0809 00:59:57.465828 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:57.465834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:57.466328 (+ 494us) server_negotiation.cc:407] Connection header received
0809 00:59:57.466618 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:57.466624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:57.466712 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:57.466817 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:57.468095 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.468923 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.470280 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.470726 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.470860 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.474482 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:57.474487 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:57.474490 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:57.474521 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:57.477597 (+ 3076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.478331 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.478336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.478339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.478471 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.478844 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.478850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.478853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.479182 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:57.479377 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:57.479826 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 00:59:57.480066 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 00:59:57.480520 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.465425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:33600)
0809 00:59:57.466170 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:57.466210 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:57.466492 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:57.466856 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:57.466860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:57.467594 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:57.467951 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.467961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.469078 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.469083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.470118 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.470129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.470936 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.470943 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.471098 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.471711 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:57.471735 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:57.474299 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:57.477731 (+ 3432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.477742 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.477748 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.478208 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.478580 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.478585 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.478588 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.478737 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.479346 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:57.479353 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:57.479537 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 00:59:57.479741 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 00:59:57.480036 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":382,"threads_started":1}
I20260809 00:59:57.481947 11810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:59:57 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237197, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.188@KRBTEST.COM for oryx/127.4.82.189@KRBTEST.COM
I20260809 00:59:57.493522 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.482611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:33311 (local address 127.0.0.1:43108)
0809 00:59:57.482783 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:57.482801 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:57.482900 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:57.483540 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:57.483543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:57.483828 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:57.483944 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.483951 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.484886 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.484889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.485746 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.485755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.486369 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.486373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.486504 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.486929 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:57.486944 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:57.488939 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:57.491604 (+ 2665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.491607 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.491610 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.491988 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.492296 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.492299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.492301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.492425 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.492925 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:57.492928 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:57.493034 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 00:59:57.493139 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 00:59:57.493322 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 00:59:57.493660 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.482858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43108 (local address 127.4.82.189:33311)
0809 00:59:57.483196 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:57.483201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:57.483217 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:59:57.483262 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:57.483266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:57.483375 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:57.483455 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:57.484064 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.484771 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.485856 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.486246 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.486335 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.489101 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:57.489105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:57.489108 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:57.489137 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:57.491501 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.492082 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.492085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.492087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.492205 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.492513 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.492516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.492518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.492813 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:57.492938 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:57.493221 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 00:59:57.493391 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221}
I20260809 00:59:57.494709 11810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:59:57 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237197, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.188@KRBTEST.COM for oryx/127.4.82.188@KRBTEST.COM
I20260809 00:59:57.506368 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.495371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42363 (local address 127.0.0.1:47882)
0809 00:59:57.495554 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:57.495570 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:57.495656 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:57.495900 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:57.495903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:57.496195 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:57.496327 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.496334 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.497154 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.497158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.498094 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.498105 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.498772 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.498776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.498927 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.499510 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:57.499525 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:57.501298 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:57.504263 (+ 2965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.504267 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.504269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.504689 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.504980 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.504982 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.504985 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.505127 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.505578 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:57.505581 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:57.505729 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 00:59:57.505872 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 00:59:57.506035 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 00:59:57.506378 11813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.495427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47882 (local address 127.4.82.188:42363)
0809 00:59:57.495576 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:57.495580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:57.495617 (+ 37us) server_negotiation.cc:407] Connection header received
0809 00:59:57.495746 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:57.495749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:57.495797 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:57.495870 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:57.496434 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.497057 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.498202 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.498654 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.498749 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.501453 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:57.501457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:57.501460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:57.501485 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:57.504151 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.504788 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.504791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.504793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.504893 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.505251 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.505254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.505256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.505485 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:57.505634 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:57.505958 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 00:59:57.506137 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":47}
I20260809 00:59:57.510931 11810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: Bootstrap starting.
I20260809 00:59:57.513931 11810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:57.514878 11810 log.cc:824] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:57.517246 11810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: No bootstrap required, opened a new log
I20260809 00:59:57.523912 11810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } }
I20260809 00:59:57.524194 11810 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:57.524277 11810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b38d7e56b6483885029e8887324a97, State: Initialized, Role: FOLLOWER
I20260809 00:59:57.524840 11810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } }
I20260809 00:59:57.525566 11816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } } }
I20260809 00:59:57.525900 11816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:59:57.526458 11810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:57.535049 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 11110
W20260809 00:59:57.535401 11828 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:59:57.535601 11828 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:59:57.540131 11647 master_runner.cc:428] Detected that this master 43b38d7e56b6483885029e8887324a97 is joining an existing cluster
I20260809 00:59:57.540254 11647 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.82.188:42363
Aug 09 00:59:57 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.82.129@KRBTEST.COM
I20260809 00:59:57.557296 11830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.536199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49238 (local address 127.4.82.129:39777)
0809 00:59:57.536809 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:57.536818 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:57.536847 (+ 29us) server_negotiation.cc:407] Connection header received
0809 00:59:57.536919 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:57.536925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:57.537149 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:57.537355 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:57.538786 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.540101 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.542020 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.542916 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.543019 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.549412 (+ 6393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:57.549429 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:57.549445 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:57.549497 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:57.553522 (+ 4025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.554423 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.554432 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.554436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.554606 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.555101 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.555108 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.555111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.555589 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:57.555868 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:57.556572 (+ 704us) server_negotiation.cc:299] Negotiation successful
0809 00:59:57.556865 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":309,"threads_started":1}
I20260809 00:59:57.559504 11238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:57.564683 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.545451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33608 (local address 127.4.82.190:44177)
0809 00:59:57.545639 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:57.545645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:57.545664 (+ 19us) server_negotiation.cc:407] Connection header received
0809 00:59:57.545733 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:57.545740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:57.545830 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:57.545934 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:57.547914 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.549440 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.552360 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.553269 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.553390 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.556324 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:57.556330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:57.556334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:57.556375 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:57.560617 (+ 4242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.561415 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.561425 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.561431 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.561680 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.562323 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.562332 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.562337 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.562769 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:57.562961 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:57.563960 (+ 999us) server_negotiation.cc:299] Negotiation successful
0809 00:59:57.564256 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 00:59:57.565155 11835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.544289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:33608)
0809 00:59:57.544982 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:57.545004 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:57.545152 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:57.546166 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:57.546174 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:57.547144 (+ 970us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:57.547664 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.547681 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.549791 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.549799 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.552132 (+ 2333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.552152 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.553756 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.553765 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.554010 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.554784 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:57.554815 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:57.556103 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:57.560809 (+ 4706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.560814 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.560818 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.561235 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.561865 (+ 630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.561872 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.561879 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.562138 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.563042 (+ 904us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:57.563050 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:57.563443 (+ 393us) client_negotiation.cc:769] Sending connection context
0809 00:59:57.563683 (+ 240us) client_negotiation.cc:240] Negotiation successful
0809 00:59:57.564548 (+ 865us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":499,"thread_start_us":1444,"threads_started":1}
I20260809 00:59:57.583887 11813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.556151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47898 (local address 127.4.82.188:42363)
0809 00:59:57.556344 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:57.556348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:57.561776 (+ 5428us) server_negotiation.cc:407] Connection header received
0809 00:59:57.562043 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:57.562050 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:57.562133 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:57.562210 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:57.567411 (+ 5201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.568605 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.573700 (+ 5095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.574539 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.574652 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.577367 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:57.577373 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:57.577378 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:57.577408 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:57.580930 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.581645 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.581650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.581653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.581795 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.582177 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.582182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.582185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.582505 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:57.582666 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:57.583269 (+ 603us) server_negotiation.cc:299] Negotiation successful
0809 00:59:57.583504 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73,"spinlock_wait_cycles":40576}
I20260809 00:59:57.584797 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.545550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43112 (local address 127.4.82.189:33311)
0809 00:59:57.545722 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:57.545728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:57.555519 (+ 9791us) server_negotiation.cc:407] Connection header received
0809 00:59:57.557017 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:57.557025 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:57.557121 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:57.557195 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:57.560099 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.561226 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.571813 (+ 10587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.572705 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.572833 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.575806 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:57.575814 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:57.575820 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:57.575867 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:57.580355 (+ 4488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.581372 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.581378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.581382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.581615 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.582303 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.582308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.582312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.582682 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:57.582959 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:57.583643 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 00:59:57.584294 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 00:59:57.586436 11836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.544933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:33311 (local address 127.0.0.1:43112)
0809 00:59:57.555407 (+ 10474us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:57.555425 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:57.556714 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:57.557485 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:57.557489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:57.558223 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:57.558592 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.558604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.562001 (+ 3397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.562009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.571635 (+ 9626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.571648 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.573137 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.573144 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.573339 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.574028 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:57.574061 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:57.575552 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:57.580651 (+ 5099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.580656 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.580662 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.581228 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.581898 (+ 670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.581905 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.581908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.582141 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.583049 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:57.583054 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:57.583270 (+ 216us) client_negotiation.cc:769] Sending connection context
0809 00:59:57.585339 (+ 2069us) client_negotiation.cc:240] Negotiation successful
0809 00:59:57.585771 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10300,"thread_start_us":10591,"threads_started":1}
I20260809 00:59:57.589982 11837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.556455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42363 (local address 127.0.0.1:47898)
0809 00:59:57.561673 (+ 5218us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:57.561693 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:57.561864 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:57.562456 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:57.562462 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:57.566850 (+ 4388us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:57.567236 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.567248 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.569443 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.569448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.571242 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.571259 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.575008 (+ 3749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.575015 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.575230 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.575899 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:57.575921 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:57.577212 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:57.581068 (+ 3856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.581073 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.581077 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.581536 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.581915 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.581920 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.581924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.582076 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.582784 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:57.582789 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:57.582936 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 00:59:57.589272 (+ 6336us) client_negotiation.cc:240] Negotiation successful
0809 00:59:57.589551 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5109,"thread_start_us":5303,"threads_started":1}
Aug 09 00:59:57 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237194, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.190@KRBTEST.COM for oryx/127.4.82.188@KRBTEST.COM
I20260809 00:59:57.600121 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.578264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42363 (local address 127.0.0.1:47910)
0809 00:59:57.578643 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:57.578665 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:57.578787 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:57.584518 (+ 5731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:57.584523 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:57.584982 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:57.585166 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.585177 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.586709 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.586725 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.588113 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.588129 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.589210 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.589219 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.589423 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.590339 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:57.590375 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:57.593225 (+ 2850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:57.597376 (+ 4151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.597382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.597387 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.597902 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.598315 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.598320 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.598324 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.598466 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.599149 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:57.599154 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:57.599340 (+ 186us) client_negotiation.cc:769] Sending connection context
0809 00:59:57.599470 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 00:59:57.599749 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233}
I20260809 00:59:57.601096 11813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.579017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47910 (local address 127.4.82.188:42363)
0809 00:59:57.584155 (+ 5138us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:57.584160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:57.584175 (+ 15us) server_negotiation.cc:407] Connection header received
0809 00:59:57.584257 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:57.584262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:57.584351 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:57.585307 (+ 956us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:57.585470 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.586560 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.588297 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.589026 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.589143 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.593400 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:57.593406 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:57.593411 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:57.593448 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:57.597225 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.598055 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.598060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.598064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.598194 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.598592 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.598597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.598601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.599001 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:57.600371 (+ 1370us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:57.600485 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 00:59:57.600689 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5051,"thread_start_us":20106,"threads_started":1}
I20260809 00:59:57.602519 11042 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.82.188:42363: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "43b38d7e56b6483885029e8887324a97"
member_type: NON_VOTER
last_known_addr {
host: "127.4.82.188"
port: 42363
}
attrs {
promote: true
}
}
dest_uuid: "c0bb0e225e8e46a8b608e6094288b96b"
cas_config_opid_index: 6
I20260809 00:59:57.603329 11042 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: NON_VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: true } }
I20260809 00:59:57.605823 11697 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 FOLLOWER]: Refusing update from remote peer c0bb0e225e8e46a8b608e6094288b96b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 00:59:57.605904 11784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:57.606485 11638 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 00:59:57.609174 11784 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 1 FOLLOWER]: Refusing update from remote peer c0bb0e225e8e46a8b608e6094288b96b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 00:59:57.609200 11639 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 43b38d7e56b6483885029e8887324a97 (127.4.82.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: NON_VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: true } } }
I20260809 00:59:57.609746 11697 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 43b38d7e56b6483885029e8887324a97 (127.4.82.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: NON_VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: true } } }
I20260809 00:59:57.613137 11639 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.82.188:42363
I20260809 00:59:57.613533 11639 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: NON_VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: true } } }
I20260809 00:59:57.613826 11647 master.cc:537] Master@127.4.82.188:42363 shutting down...
I20260809 00:59:57.613857 11639 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:57.615314 11640 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: NON_VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: true } } }
I20260809 00:59:57.615603 11640 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:57.617062 11727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: NON_VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: true } } }
I20260809 00:59:57.617385 11727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:59:57.619639 11638 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: NON_VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 00:59:57.623112 11026 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 00:59:57.628558 11026 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b -> Peer 43b38d7e56b6483885029e8887324a97 (127.4.82.188:42363): Couldn't send request to peer 43b38d7e56b6483885029e8887324a97. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 00:59:57.649564 11647 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:57.650027 11647 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:57.650247 11647 tablet_replica.cc:330] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: stopping tablet replica
I20260809 00:59:57.662353 11647 master.cc:552] Master@127.4.82.188:42363 shutdown complete.
I20260809 00:59:57.662451 11647 master_runner.cc:305] Clearing existing system tablet
I20260809 00:59:57.662878 11647 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:57.663005 11647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:57.675900 11647 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.004s sys 0.008s
I20260809 00:59:57.677542 11843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:57.677913 11647 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:57.678174 11647 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:59:57.678275 11647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/wal
uuid: "43b38d7e56b6483885029e8887324a97"
format_stamp: "Formatted at 2026-08-09 00:59:57 on dist-test-slave-07c2"
server_key: "b8256b459693caf18ad69007cfc2c5a2"
server_key_iv: "99aeca865814af8f05b3920cfe5ca03f"
server_key_version: "encryptionkey@0"
I20260809 00:59:57.683266 11647 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:57.690130 11647 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 00:59:57.690286 11647 log.cc:1196] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 00:59:57.690747 11647 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: Deleting consensus metadata
I20260809 00:59:57.691016 11647 master_runner.cc:315] Copying system tablet from 127.4.82.190:44177
I20260809 00:59:57.694433 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 11110
I20260809 00:59:57.708853 11647 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: tablet copy: Beginning tablet copy session from remote peer at address 127.4.82.190:44177
I20260809 00:59:57.717818 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:39777
--local_ip_for_outbound_sockets=127.4.82.129
--tserver_master_addrs=127.4.82.190:44177
--webserver_port=35463
--webserver_interface=127.4.82.129
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab
--principal=oryx/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.82.190:44177,127.4.82.189:33311
--tserver_master_addrs=127.4.82.190:44177,127.4.82.189:33311,127.4.82.188:42363 with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:59:57.731118 11849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.709701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:33616)
0809 00:59:57.716167 (+ 6466us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:57.716197 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:57.716352 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:57.716808 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:57.716814 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:57.717343 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:57.717608 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.717625 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.719151 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.719157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.720712 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:57.720727 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.721758 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.721764 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:57.721963 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.722704 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:57.722733 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:57.723827 (+ 1094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:57.728108 (+ 4281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.728114 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.728118 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.728661 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.729090 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:57.729095 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:57.729099 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:57.729304 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:57.730039 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:57.730044 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:57.730216 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 00:59:57.730356 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 00:59:57.730672 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6318,"thread_start_us":6191,"threads_started":1}
I20260809 00:59:57.731199 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:57.709727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33616 (local address 127.4.82.190:44177)
0809 00:59:57.709958 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:57.709965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:57.716307 (+ 6342us) server_negotiation.cc:407] Connection header received
0809 00:59:57.716489 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:57.716497 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:57.716591 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:57.716702 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:57.717801 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.718977 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.720866 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:57.721578 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:57.721726 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:57.724049 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:57.724057 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:57.724064 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:57.724107 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:57.727946 (+ 3839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.728805 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.728814 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.728820 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.728977 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:57.729448 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:57.729455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:57.729461 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:57.729919 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:57.730075 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:57.730514 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 00:59:57.730781 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260809 00:59:57.732146 11065 tablet_copy_service.cc:140] P c0bb0e225e8e46a8b608e6094288b96b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 43b38d7e56b6483885029e8887324a97 ({username='oryx', principal='oryx/127.4.82.188@KRBTEST.COM'} at 127.0.0.1:33616)
I20260809 00:59:57.732342 11065 tablet_copy_service.cc:161] P c0bb0e225e8e46a8b608e6094288b96b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 43b38d7e56b6483885029e8887324a97 at {username='oryx', principal='oryx/127.4.82.188@KRBTEST.COM'} at 127.0.0.1:33616: session id = 43b38d7e56b6483885029e8887324a97-00000000000000000000000000000000
I20260809 00:59:57.736500 11065 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 00:59:57.738425 11647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 00:59:57.745880 11647 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: tablet copy: Starting download of 0 data blocks...
I20260809 00:59:57.746173 11647 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: tablet copy: Starting download of 1 WAL segments...
I20260809 00:59:57.752151 11647 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 00:59:57.757279 11065 tablet_copy_service.cc:342] P c0bb0e225e8e46a8b608e6094288b96b: Request end of tablet copy session 43b38d7e56b6483885029e8887324a97-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.82.188@KRBTEST.COM'} at 127.0.0.1:33616
I20260809 00:59:57.757423 11065 tablet_copy_service.cc:433] P c0bb0e225e8e46a8b608e6094288b96b: ending tablet copy session 43b38d7e56b6483885029e8887324a97-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 43b38d7e56b6483885029e8887324a97
I20260809 00:59:57.768981 11647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:57.771538 11851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:57.776602 11852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:57.783481 11854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:57.792640 11647 server_base.cc:1034] running on GCE node
I20260809 00:59:57.793047 11647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:57.793733 11647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:57.795002 11647 hybrid_clock.cc:648] HybridClock initialized: now 1786237197794959 us; error 35 us; skew 500 ppm
I20260809 00:59:57.795265 11647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:57 dist-test-slave-07c2 krb5kdc[10995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237197, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:57.797500 11647 init.cc:377] Logged in from keytab as oryx/127.4.82.188@KRBTEST.COM (short username oryx)
I20260809 00:59:57.804312 11647 webserver.cc:460] Webserver started at http://127.4.82.188:35211/ using document root <none> and password file <none>
I20260809 00:59:57.804642 11647 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:57.804766 11647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:57.807847 11647 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 00:59:57.809917 11862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:57.810410 11647 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:59:57.810542 11647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/wal
uuid: "43b38d7e56b6483885029e8887324a97"
format_stamp: "Formatted at 2026-08-09 00:59:57 on dist-test-slave-07c2"
server_key: "b8256b459693caf18ad69007cfc2c5a2"
server_key_iv: "99aeca865814af8f05b3920cfe5ca03f"
server_key_version: "encryptionkey@0"
I20260809 00:59:57.810721 11647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:57.837189 11647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:57.841348 11647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:57.867560 11647 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.188:42363
I20260809 00:59:57.867586 11917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.188:42363 every 8 connection(s)
I20260809 00:59:57.868588 11647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/master-2/data/info.pb
I20260809 00:59:57.874927 11918 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:59:57.878508 11918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: Bootstrap starting.
I20260809 00:59:57.898396 11918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:59:57.899051 11918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: Bootstrap complete.
I20260809 00:59:57.900333 11918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: NON_VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: true } }
I20260809 00:59:57.900664 11918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 43b38d7e56b6483885029e8887324a97, State: Initialized, Role: LEARNER
I20260809 00:59:57.900918 11918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: NON_VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: true } }
I20260809 00:59:57.902033 11918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:59:57.901809 11922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: NON_VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: true } } }
I20260809 00:59:57.902304 11922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: This master's current role is: LEARNER
I20260809 00:59:57.911708 11934 catalog_manager.cc:1259] Loaded cluster ID: 91c290b0219847f5a904518c3570ed7b
I20260809 00:59:57.911801 11934 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: loading cluster ID for follower catalog manager: success
I20260809 00:59:57.915230 11934 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: acquiring CA information for follower catalog manager: success
I20260809 00:59:57.917568 11934 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 00:59:58.091995 11850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:58.092291 11850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:58.092463 11850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:58.100330 11850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:58.100467 11850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:58.100656 11850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 00:59:58.110227 11850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab
--principal=oryx/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:39777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=35463
--tserver_master_addrs=127.4.82.190:44177,127.4.82.189:33311,127.4.82.188:42363
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:58.111481 11850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:58.113001 11850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:58.125945 11940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:58.125921 11942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:58.128867 11850 server_base.cc:1034] running on GCE node
W20260809 00:59:58.129522 11939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:58.130746 11850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:58.132068 11850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:58.133287 11850 hybrid_clock.cc:648] HybridClock initialized: now 1786237198133237 us; error 42 us; skew 500 ppm
I20260809 00:59:58.133610 11850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:58 dist-test-slave-07c2 krb5kdc[10995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237198, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:58.136924 11850 init.cc:377] Logged in from keytab as oryx/127.4.82.129@KRBTEST.COM (short username oryx)
I20260809 00:59:58.137075 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.129504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47914 (local address 127.4.82.188:42363)
0809 00:59:58.130074 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:58.130078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:58.131699 (+ 1621us) server_negotiation.cc:407] Connection header received
0809 00:59:58.131913 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:58.131919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:58.131998 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:58.132121 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:58.132616 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.133683 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.135202 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.136059 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.136195 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.136437 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:58.136596 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 00:59:58.136744 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":308,"threads_started":1}
I20260809 00:59:58.137101 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.129224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42363 (local address 127.0.0.1:47914)
0809 00:59:58.131589 (+ 2365us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:58.131608 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:58.131773 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:58.132195 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:58.132203 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:58.132234 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:58.132439 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.132450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.133841 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.133846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.135043 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.135064 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.136212 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.136218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.136405 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.136444 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 00:59:58.136567 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 00:59:58.136732 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2272,"thread_start_us":2178,"threads_started":1}
I20260809 00:59:58.138121 11891 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260809 00:59:58.138648 11922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0bb0e225e8e46a8b608e6094288b96b. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: NON_VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: true } } }
I20260809 00:59:58.138873 11922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: This master's current role is: LEARNER
I20260809 00:59:58.144987 11850 webserver.cc:460] Webserver started at http://127.4.82.129:35463/ using document root <none> and password file <none>
I20260809 00:59:58.145843 11850 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:58.145980 11850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:58.152619 11850 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 00:59:58.155714 11952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:58.157250 11850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 00:59:58.157413 11850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "3d14b4ca8b0140d2a29afca36f844a53"
format_stamp: "Formatted at 2026-08-09 00:59:55 on dist-test-slave-07c2"
server_key: "8d9fcc7b24c64cc2400338b8a2db44fd"
server_key_iv: "14b397413b2688e7f0871fd3021bde34"
server_key_version: "encryptionkey@0"
I20260809 00:59:58.158352 11850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:58.184270 11850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:58.188246 11850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:58.188529 11850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:58.189551 11850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:58.190901 11850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:58.191015 11850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:58.191139 11850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:58.191244 11850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:58.226758 11850 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:39777
I20260809 00:59:58.226862 12073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:39777 every 8 connection(s)
I20260809 00:59:58.228376 11850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 00:59:58.236363 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 11850
I20260809 00:59:58.236714 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 11251
Aug 09 00:59:58 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237198, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.188@KRBTEST.COM
Aug 09 00:59:58 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237198, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.189@KRBTEST.COM
Aug 09 00:59:58 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237198, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 00:59:58.258764 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.231764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:34003 (local address 127.4.82.188:42363)
0809 00:59:58.231922 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:58.231926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:58.233232 (+ 1306us) server_negotiation.cc:407] Connection header received
0809 00:59:58.236118 (+ 2886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:58.236124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:58.236198 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:58.236288 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:58.239273 (+ 2985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.240468 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.244115 (+ 3647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.244779 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.244889 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.250134 (+ 5245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:58.250140 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:58.250145 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:58.250195 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:58.254717 (+ 4522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.256327 (+ 1610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.256332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.256336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.256491 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.256977 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.256982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.256985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.257370 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:58.257518 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:58.258129 (+ 611us) server_negotiation.cc:299] Negotiation successful
0809 00:59:58.258392 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:59:58.258983 12081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.234075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:44187 (local address 127.4.82.189:33311)
0809 00:59:58.235631 (+ 1556us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:58.235635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:58.235655 (+ 20us) server_negotiation.cc:407] Connection header received
0809 00:59:58.236131 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:58.236138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:58.236226 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:58.236944 (+ 718us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:58.239636 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.241158 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.244009 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.244675 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.244814 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.251062 (+ 6248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:58.251067 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:58.251070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:58.251102 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:58.254717 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.255786 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.255792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.255797 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.255919 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.256436 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.256441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.256444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.256782 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:58.256932 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:58.258368 (+ 1436us) server_negotiation.cc:299] Negotiation successful
0809 00:59:58.258614 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1464,"thread_start_us":1342,"threads_started":1}
I20260809 00:59:58.260953 12080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.233842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:33311 (local address 127.4.82.129:44187)
0809 00:59:58.234432 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:58.234445 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:58.235770 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:58.236631 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:58.236641 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:58.237850 (+ 1209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:58.239001 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.239048 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.241367 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.241375 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.243877 (+ 2502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.243892 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.244925 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.244932 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.245218 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.246255 (+ 1037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:58.246289 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:58.250825 (+ 4536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:58.254883 (+ 4058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:58.254892 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:58.254910 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:58.255670 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:58.256035 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:58.256039 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:58.256042 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:58.256296 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:58.257010 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:58.257021 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:58.257426 (+ 405us) client_negotiation.cc:769] Sending connection context
0809 00:59:58.257773 (+ 347us) client_negotiation.cc:240] Negotiation successful
0809 00:59:58.259521 (+ 1748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":315,"spinlock_wait_cycles":15488,"thread_start_us":330,"threads_started":1}
I20260809 00:59:58.260984 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.232469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42363 (local address 127.4.82.129:34003)
0809 00:59:58.233112 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:58.233132 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:58.235727 (+ 2595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:58.236629 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:58.236640 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:58.238524 (+ 1884us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:58.239001 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.239048 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.240643 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.240647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.243426 (+ 2779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.243446 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.245140 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.245144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.245261 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.246147 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:58.246189 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:58.249894 (+ 3705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:58.254883 (+ 4989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:58.254892 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:58.254910 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:58.256178 (+ 1268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:58.256640 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:58.256646 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:58.256650 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:58.256830 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:58.257589 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:58.257596 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:58.257762 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 00:59:58.257930 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 00:59:58.259748 (+ 1818us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":1559,"spinlock_wait_cycles":47104,"thread_start_us":978,"threads_started":1}
I20260809 00:59:58.262526 12078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.232068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.129:56749)
0809 00:59:58.232834 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:58.232875 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:58.235727 (+ 2852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:58.236629 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:58.236640 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:58.238138 (+ 1498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:58.239432 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.239447 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.244826 (+ 5379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.244831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.247450 (+ 2619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.247465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.251081 (+ 3616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.251090 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.251371 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.251994 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:58.252024 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:58.255605 (+ 3581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:58.259630 (+ 4025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:58.259634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:58.259637 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:58.260074 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:58.260511 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:58.260516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:58.260519 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:58.260726 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:58.261466 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:58.261470 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:58.261695 (+ 225us) client_negotiation.cc:769] Sending connection context
0809 00:59:58.261881 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 00:59:58.262141 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":6,"thread_start_us":495,"threads_started":1}
I20260809 00:59:58.262653 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.231455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:56749 (local address 127.4.82.190:44177)
0809 00:59:58.231690 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:58.231696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:58.233004 (+ 1308us) server_negotiation.cc:407] Connection header received
0809 00:59:58.236047 (+ 3043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:58.236056 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:58.236145 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:58.236234 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:58.239570 (+ 3336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.240652 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.248708 (+ 8056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.249602 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.249768 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.255849 (+ 6081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:58.255856 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:58.255861 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:58.255901 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:58.259489 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.260219 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.260224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.260227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.260369 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.260857 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.260864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.260870 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.261239 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:58.261377 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:58.261998 (+ 621us) server_negotiation.cc:299] Negotiation successful
0809 00:59:58.262242 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 00:59:58.262777 12074 heartbeater.cc:347] Connected to a master server at 127.4.82.188:42363
I20260809 00:59:58.262895 12076 heartbeater.cc:347] Connected to a master server at 127.4.82.189:33311
I20260809 00:59:58.263134 12074 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:58.263139 12076 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:58.264102 12076 heartbeater.cc:511] Master 127.4.82.189:33311 requested a full tablet report, sending...
I20260809 00:59:58.264166 12074 heartbeater.cc:511] Master 127.4.82.188:42363 requested a full tablet report, sending...
I20260809 00:59:58.264698 12075 heartbeater.cc:347] Connected to a master server at 127.4.82.190:44177
I20260809 00:59:58.264825 12075 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:58.265231 12075 heartbeater.cc:511] Master 127.4.82.190:44177 requested a full tablet report, sending...
I20260809 00:59:58.266300 11042 ts_manager.cc:194] Re-registered known tserver with Master: 3d14b4ca8b0140d2a29afca36f844a53 (127.4.82.129:39777)
I20260809 00:59:58.266703 11685 ts_manager.cc:194] Registered new tserver with Master: 3d14b4ca8b0140d2a29afca36f844a53 (127.4.82.129:39777)
I20260809 00:59:58.266708 11880 ts_manager.cc:194] Registered new tserver with Master: 3d14b4ca8b0140d2a29afca36f844a53 (127.4.82.129:39777)
I20260809 00:59:58.267355 11042 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:56749
Aug 09 00:59:58 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.82.130@KRBTEST.COM
I20260809 00:59:58.277907 12083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.237798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59710 (local address 127.4.82.130:42733)
0809 00:59:58.243827 (+ 6029us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:58.243832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:58.256433 (+ 12601us) server_negotiation.cc:407] Connection header received
0809 00:59:58.256513 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:58.256525 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:58.256787 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:58.256937 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:58.259198 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.260719 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.264001 (+ 3282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.264851 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.264983 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.272485 (+ 7502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:58.272496 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:58.272506 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:58.272540 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:58.275353 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.276033 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.276039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.276041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.276181 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.276487 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.276490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.276492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.276782 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:58.276995 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:58.277255 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 00:59:58.277504 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5890,"thread_start_us":5734,"threads_started":1}
I20260809 00:59:58.279099 11381 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:58.362742 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 11251
I20260809 00:59:58.381255 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:42733
--local_ip_for_outbound_sockets=127.4.82.130
--tserver_master_addrs=127.4.82.190:44177
--webserver_port=39793
--webserver_interface=127.4.82.130
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab
--principal=oryx/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.82.190:44177,127.4.82.189:33311
--tserver_master_addrs=127.4.82.190:44177,127.4.82.189:33311,127.4.82.188:42363 with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:59:58.710464 12085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:58.710749 12085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:58.710875 12085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:58.718065 12085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:58.718179 12085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:58.718286 12085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 00:59:58.725373 11638 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: NON_VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: true } } }
I20260809 00:59:58.725744 11638 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:58.726929 12090 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b: attempting to promote NON_VOTER 43b38d7e56b6483885029e8887324a97 to VOTER
I20260809 00:59:58.727669 12090 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } }
I20260809 00:59:58.728896 12085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab
--principal=oryx/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:42733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=39793
--tserver_master_addrs=127.4.82.190:44177,127.4.82.189:33311,127.4.82.188:42363
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:58.729717 11891 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 1 LEARNER]: Refusing update from remote peer c0bb0e225e8e46a8b608e6094288b96b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 00:59:58.730223 11697 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 FOLLOWER]: Refusing update from remote peer c0bb0e225e8e46a8b608e6094288b96b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 00:59:58.730286 12090 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [LEADER]: Connected to new peer: Peer: permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 00:59:58.730633 12085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:58.730893 12089 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 00:59:58.732374 12085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:59:58.734272 11638 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 43b38d7e56b6483885029e8887324a97 (127.4.82.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } } }
I20260809 00:59:58.734894 11697 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 43b38d7e56b6483885029e8887324a97 (127.4.82.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } } }
I20260809 00:59:58.734923 11891 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 43b38d7e56b6483885029e8887324a97 (127.4.82.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } } }
I20260809 00:59:58.738405 12090 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } } }
I20260809 00:59:58.738713 12090 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:58.739102 11638 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } } }
I20260809 00:59:58.739573 11638 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:58.739629 12089 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } } }
I20260809 00:59:58.739915 12089 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b [sys.catalog]: This master's current role is: LEADER
I20260809 00:59:58.747656 12095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } } }
I20260809 00:59:58.748190 12095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:59:58.748847 12097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0bb0e225e8e46a8b608e6094288b96b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } } }
I20260809 00:59:58.749289 12097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:59:58.752606 12100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:58.752954 12102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:58.753008 12099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:58.753664 12085 server_base.cc:1034] running on GCE node
I20260809 00:59:58.754248 12085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:58.755124 12085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:58.756356 12085 hybrid_clock.cc:648] HybridClock initialized: now 1786237198756308 us; error 40 us; skew 500 ppm
I20260809 00:59:58.756664 12085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:58 dist-test-slave-07c2 krb5kdc[10995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237198, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:58.759750 12085 init.cc:377] Logged in from keytab as oryx/127.4.82.130@KRBTEST.COM (short username oryx)
I20260809 00:59:58.761091 12085 webserver.cc:460] Webserver started at http://127.4.82.130:39793/ using document root <none> and password file <none>
I20260809 00:59:58.761785 12085 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:58.761943 12085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:58.767133 12085 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 00:59:58.769878 12109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:58.771104 12085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 00:59:58.771276 12085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "27c702680b924d28995faa26a770ed54"
format_stamp: "Formatted at 2026-08-09 00:59:55 on dist-test-slave-07c2"
server_key: "8d7104dcb8a4e5f694ba43ff5dba8672"
server_key_iv: "7154fd896f07875571a0e1c93c244fe6"
server_key_version: "encryptionkey@0"
I20260809 00:59:58.772262 12085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:58.788303 12085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:58.792063 12085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:58.792320 12085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:58.793345 12085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:58.794697 12085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:58.794816 12085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:58.794939 12085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:58.795030 12085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:58.827895 12085 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:42733
I20260809 00:59:58.827975 12230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:42733 every 8 connection(s)
I20260809 00:59:58.829275 12085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 00:59:58.834395 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 12085
I20260809 00:59:58.834748 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 11394
Aug 09 00:59:58 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237198, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
Aug 09 00:59:58 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237198, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.188@KRBTEST.COM
Aug 09 00:59:58 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.82.131@KRBTEST.COM
Aug 09 00:59:58 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237198, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.189@KRBTEST.COM
I20260809 00:59:58.874948 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.833459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:51737 (local address 127.4.82.190:44177)
0809 00:59:58.848053 (+ 14594us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:58.848062 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:58.852581 (+ 4519us) server_negotiation.cc:407] Connection header received
0809 00:59:58.855010 (+ 2429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:58.855015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:58.855102 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:58.855194 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:58.857739 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.858498 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.861063 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.861747 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.861845 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.867151 (+ 5306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:58.867169 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:58.867174 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:58.867214 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:58.871338 (+ 4124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.872272 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.872280 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.872284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.872459 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.872967 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.872972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.872976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.873301 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:58.873433 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:58.874355 (+ 922us) server_negotiation.cc:299] Negotiation successful
0809 00:59:58.874566 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14436,"thread_start_us":14330,"threads_started":1}
I20260809 00:59:58.876128 12240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.835269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:49003 (local address 127.4.82.188:42363)
0809 00:59:58.851771 (+ 16502us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:58.851780 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:58.853866 (+ 2086us) server_negotiation.cc:407] Connection header received
0809 00:59:58.855009 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:58.855015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:58.855094 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:58.855195 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:58.859157 (+ 3962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.860297 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.862657 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.863349 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.863436 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.867703 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:58.867729 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:58.867735 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:58.867779 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:58.871943 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.872811 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.872819 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.872825 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.872964 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.873412 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.873417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.873420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.873796 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:58.873925 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:58.875429 (+ 1504us) server_negotiation.cc:299] Negotiation successful
0809 00:59:58.875641 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16359,"thread_start_us":16188,"threads_started":1}
I20260809 00:59:58.876192 12235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.832041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:59359 (local address 127.4.82.189:33311)
0809 00:59:58.847621 (+ 15580us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:58.847631 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:58.852651 (+ 5020us) server_negotiation.cc:407] Connection header received
0809 00:59:58.854951 (+ 2300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:58.854959 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:58.855043 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:58.855130 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:58.859117 (+ 3987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.860269 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.862657 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.863338 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.863436 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.869526 (+ 6090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:58.869533 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:58.869537 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:58.869578 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:58.873152 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.873929 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.873935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.873941 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.874081 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.874528 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.874533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.874536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.874910 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:58.875028 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:58.875618 (+ 590us) server_negotiation.cc:299] Negotiation successful
0809 00:59:58.875828 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15408,"thread_start_us":16023,"threads_started":1}
I20260809 00:59:58.876688 12239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.833537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.130:51737)
0809 00:59:58.852357 (+ 18820us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:58.852386 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:58.854635 (+ 2249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:58.855594 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:58.855605 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:58.856613 (+ 1008us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:58.857562 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.857586 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.858699 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.858705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.860881 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.860896 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.862196 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.862205 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.862582 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.863352 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:58.863383 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:58.866883 (+ 3500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:58.871503 (+ 4620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:58.871511 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:58.871525 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:58.872112 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:58.872619 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:58.872623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:58.872627 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:58.872848 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:58.873533 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:58.873540 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:58.873932 (+ 392us) client_negotiation.cc:769] Sending connection context
0809 00:59:58.875203 (+ 1271us) client_negotiation.cc:240] Negotiation successful
0809 00:59:58.875484 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18629,"mutex_wait_us":183,"spinlock_wait_cycles":23424,"thread_start_us":19936,"threads_started":1}
I20260809 00:59:58.877260 12236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.832783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:33311 (local address 127.4.82.130:59359)
0809 00:59:58.833796 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:58.851390 (+ 17594us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:58.854618 (+ 3228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:58.855596 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:58.855605 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:58.856587 (+ 982us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:58.858919 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.858938 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.860462 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.860467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.862449 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.862463 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.863738 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.863744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.863938 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.864524 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:58.864552 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:58.869265 (+ 4713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:58.873318 (+ 4053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:58.873325 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:58.873330 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:58.873794 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:58.874235 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:58.874239 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:58.874242 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:58.874410 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:58.875128 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:58.875132 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:58.875305 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 00:59:58.876452 (+ 1147us) client_negotiation.cc:240] Negotiation successful
0809 00:59:58.876781 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"mutex_wait_us":1665,"thread_start_us":844,"threads_started":1}
I20260809 00:59:58.878651 12232 heartbeater.cc:347] Connected to a master server at 127.4.82.190:44177
I20260809 00:59:58.878976 12232 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:58.879191 12238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.834845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42363 (local address 127.4.82.130:49003)
0809 00:59:58.853757 (+ 18912us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:58.853780 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:58.854635 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:58.855594 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:58.855605 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:58.856608 (+ 1003us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:58.858991 (+ 2383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.859004 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.860462 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.860467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.862449 (+ 1982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:58.862463 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.864037 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.864046 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:58.864250 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.864827 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:58.864856 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:58.867458 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:58.872097 (+ 4639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:58.872104 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:58.872111 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:58.872647 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:58.873101 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:58.873105 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:58.873109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:58.873280 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:58.873994 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:58.874002 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:58.874161 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 00:59:58.874424 (+ 263us) client_negotiation.cc:240] Negotiation successful
0809 00:59:58.874910 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18790,"mutex_wait_us":1637,"thread_start_us":17106,"threads_started":1}
I20260809 00:59:58.879426 12233 heartbeater.cc:347] Connected to a master server at 127.4.82.189:33311
I20260809 00:59:58.879581 12233 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:58.880048 12232 heartbeater.cc:511] Master 127.4.82.190:44177 requested a full tablet report, sending...
I20260809 00:59:58.880102 12233 heartbeater.cc:511] Master 127.4.82.189:33311 requested a full tablet report, sending...
I20260809 00:59:58.880578 12231 heartbeater.cc:347] Connected to a master server at 127.4.82.188:42363
I20260809 00:59:58.880705 12231 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:58.881194 12231 heartbeater.cc:511] Master 127.4.82.188:42363 requested a full tablet report, sending...
I20260809 00:59:58.881748 11042 ts_manager.cc:194] Re-registered known tserver with Master: 27c702680b924d28995faa26a770ed54 (127.4.82.130:42733)
I20260809 00:59:58.882426 11685 ts_manager.cc:194] Registered new tserver with Master: 27c702680b924d28995faa26a770ed54 (127.4.82.130:42733)
I20260809 00:59:58.882835 11042 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:51737
I20260809 00:59:58.883415 11880 ts_manager.cc:194] Registered new tserver with Master: 27c702680b924d28995faa26a770ed54 (127.4.82.130:42733)
I20260809 00:59:58.887406 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:58.836354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46514 (local address 127.4.82.131:39315)
0809 00:59:58.853265 (+ 16911us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:58.853277 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:58.854382 (+ 1105us) server_negotiation.cc:407] Connection header received
0809 00:59:58.854499 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:58.854508 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:58.854760 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:58.854906 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:58.857240 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.858396 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.860340 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:58.861171 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:58.861289 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:58.877766 (+ 16477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:58.877782 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:58.877797 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:58.877843 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:58.884658 (+ 6815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.885501 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.885507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.885509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.885643 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:58.885987 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:58.885990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:58.885992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:58.886299 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:58.886517 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:58.886765 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 00:59:58.886952 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16683,"thread_start_us":16489,"threads_started":1}
I20260809 00:59:58.888513 11524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:58.972316 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 11394
I20260809 00:59:58.991766 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:39315
--local_ip_for_outbound_sockets=127.4.82.131
--tserver_master_addrs=127.4.82.190:44177
--webserver_port=33957
--webserver_interface=127.4.82.131
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab
--principal=oryx/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:34117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.82.190:44177,127.4.82.189:33311
--tserver_master_addrs=127.4.82.190:44177,127.4.82.189:33311,127.4.82.188:42363 with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:59:59.270063 12075 heartbeater.cc:503] Master 127.4.82.190:44177 was elected leader, sending a full tablet report...
W20260809 00:59:59.336807 12244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:59:59.337124 12244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:59:59.337287 12244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:59:59.344874 12244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:59:59.344973 12244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:59:59.345176 12244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 00:59:59.354712 12244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab
--principal=oryx/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:39315
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=33957
--tserver_master_addrs=127.4.82.190:44177,127.4.82.189:33311,127.4.82.188:42363
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 00:59:59.356041 12244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:59:59.357314 12244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:59:59.371052 12249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:59.371259 12253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:59:59.371364 12251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:59:59.372282 12244 server_base.cc:1034] running on GCE node
I20260809 00:59:59.372866 12244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:59:59.373881 12244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:59:59.375077 12244 hybrid_clock.cc:648] HybridClock initialized: now 1786237199375037 us; error 52 us; skew 500 ppm
I20260809 00:59:59.375470 12244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 00:59:59 dist-test-slave-07c2 krb5kdc[10995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237199, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:59:59.378937 12244 init.cc:377] Logged in from keytab as oryx/127.4.82.131@KRBTEST.COM (short username oryx)
I20260809 00:59:59.380326 12244 webserver.cc:460] Webserver started at http://127.4.82.131:33957/ using document root <none> and password file <none>
I20260809 00:59:59.380995 12244 fs_manager.cc:359] Metadata directory not provided
I20260809 00:59:59.381151 12244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:59:59.386241 12244 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 00:59:59.389032 12260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:59.390265 12244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 00:59:59.390439 12244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "964f6e9dde52455299287e03815ae164"
format_stamp: "Formatted at 2026-08-09 00:59:56 on dist-test-slave-07c2"
server_key: "082a6b99e0d05d41870fb4b7d8ebe191"
server_key_iv: "9481e1631628df53af6668bfe2fd09fb"
server_key_version: "encryptionkey@0"
I20260809 00:59:59.391438 12244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:59:59.416198 12244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:59:59.419931 12244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:59:59.420212 12244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:59:59.421159 12244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:59:59.422469 12244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:59:59.422585 12244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:59.422722 12244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:59:59.422821 12244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:59:59.457976 12244 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:39315
I20260809 00:59:59.458103 12384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:39315 every 8 connection(s)
I20260809 00:59:59.459465 12244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 00:59:59.465054 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 12244
Aug 09 00:59:59 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237199, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.189@KRBTEST.COM
Aug 09 00:59:59 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237199, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.188@KRBTEST.COM
Aug 09 00:59:59 dist-test-slave-07c2 krb5kdc[10995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237199, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 00:59:59.509267 12396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.468976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42363 (local address 127.4.82.131:36579)
0809 00:59:59.479775 (+ 10799us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:59.479804 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:59.479955 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:59.480923 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:59.480937 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:59.481992 (+ 1055us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:59.483004 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.483030 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.484660 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.484665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.487191 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.487206 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.488500 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.488505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.488757 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.489828 (+ 1071us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:59.489866 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:59.498140 (+ 8274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:59.503125 (+ 4985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:59.503130 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:59.503136 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:59.503660 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:59.504700 (+ 1040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:59.504704 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:59.504707 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:59.504976 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:59.506129 (+ 1153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:59.506139 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:59.506624 (+ 485us) client_negotiation.cc:769] Sending connection context
0809 00:59:59.507005 (+ 381us) client_negotiation.cc:240] Negotiation successful
0809 00:59:59.507649 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10642,"mutex_wait_us":163,"spinlock_wait_cycles":19456,"thread_start_us":10504,"threads_started":1}
I20260809 00:59:59.510390 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.468036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:36579 (local address 127.4.82.188:42363)
0809 00:59:59.480397 (+ 12361us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.480402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.480418 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:59:59.480468 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.480472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.480529 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.480658 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:59.483229 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.484467 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.487387 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.488077 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.488204 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.499411 (+ 11207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:59.499421 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:59.499430 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:59.499483 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:59.502981 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.503837 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.503844 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.503849 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.503996 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.505139 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.505146 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.505152 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.505597 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:59.505773 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.509771 (+ 3998us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.510028 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12249,"thread_start_us":12140,"threads_started":1}
I20260809 00:59:59.512096 12385 heartbeater.cc:347] Connected to a master server at 127.4.82.188:42363
I20260809 00:59:59.512418 12385 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:59.513186 12390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.462405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:42673 (local address 127.4.82.189:33311)
0809 00:59:59.481138 (+ 18733us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.481147 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.481168 (+ 21us) server_negotiation.cc:407] Connection header received
0809 00:59:59.481240 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.481246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.481319 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.483172 (+ 1853us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:59.483367 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.484452 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.487386 (+ 2934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.488040 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.489174 (+ 1134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.495983 (+ 6809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:59.495992 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:59.496018 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:59.496122 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:59.502103 (+ 5981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.503161 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.503179 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.503185 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.503381 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.508334 (+ 4953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.508339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.508341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.508724 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:59.508868 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.512351 (+ 3483us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.512649 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18599,"thread_start_us":18498,"threads_started":1}
I20260809 00:59:59.513403 12385 heartbeater.cc:511] Master 127.4.82.188:42363 requested a full tablet report, sending...
I20260809 00:59:59.515885 11880 ts_manager.cc:194] Registered new tserver with Master: 964f6e9dde52455299287e03815ae164 (127.4.82.131:39315)
I20260809 00:59:59.517493 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.463810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:33311 (local address 127.4.82.131:42673)
0809 00:59:59.467703 (+ 3893us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:59.467727 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:59.472246 (+ 4519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:59.481506 (+ 9260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:59.481510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:59.482142 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:59.483004 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.483030 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.484662 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.484665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.487191 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.487206 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.488390 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.488398 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.488721 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.489709 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:59.489748 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:59.495684 (+ 5936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:59.502273 (+ 6589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:59.502282 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:59.502301 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:59.502977 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:59.508003 (+ 5026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:59.508008 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:59.508011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:59.508188 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:59.511242 (+ 3054us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:59.511250 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:59.511472 (+ 222us) client_negotiation.cc:769] Sending connection context
0809 00:59:59.511667 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 00:59:59.512044 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3755,"mutex_wait_us":353,"spinlock_wait_cycles":1408,"thread_start_us":3611,"threads_started":1}
I20260809 00:59:59.518535 12387 heartbeater.cc:347] Connected to a master server at 127.4.82.189:33311
I20260809 00:59:59.518654 12387 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:59.519170 12387 heartbeater.cc:511] Master 127.4.82.189:33311 requested a full tablet report, sending...
I20260809 00:59:59.520608 11685 ts_manager.cc:194] Registered new tserver with Master: 964f6e9dde52455299287e03815ae164 (127.4.82.131:39315)
I20260809 00:59:59.521675 12391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.463156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.131:52561)
0809 00:59:59.464173 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:59.464222 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:59.472615 (+ 8393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:59.490015 (+ 17400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:59.490022 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:59.490616 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:59:59.490916 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.490931 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.492467 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.492472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.494209 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.494224 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.497273 (+ 3049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.497282 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.497604 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.498239 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:59:59.498266 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:59:59.502208 (+ 3942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:59:59.516885 (+ 14677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:59.516892 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:59.516896 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:59.517389 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:59.519591 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:59:59.519598 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:59:59.519602 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:59:59.519831 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:59:59.520549 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:59:59.520554 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:59:59.520756 (+ 202us) client_negotiation.cc:769] Sending connection context
0809 00:59:59.520926 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 00:59:59.521241 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":670,"threads_started":1}
I20260809 00:59:59.523233 12389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.462405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:52561 (local address 127.4.82.190:44177)
0809 00:59:59.489611 (+ 27206us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.489620 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.489641 (+ 21us) server_negotiation.cc:407] Connection header received
0809 00:59:59.489713 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.489718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.489830 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.490503 (+ 673us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:59.491111 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.492306 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.495155 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.496869 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.497033 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.503582 (+ 6549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:59.503589 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:59.503595 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:59.503638 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:59.516726 (+ 13088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.519198 (+ 2472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.519206 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.519212 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.519438 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.519981 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.519988 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.519994 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.520415 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:59.520596 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.521427 (+ 831us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.521758 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27040,"thread_start_us":26907,"threads_started":1}
I20260809 00:59:59.524123 12386 heartbeater.cc:347] Connected to a master server at 127.4.82.190:44177
I20260809 00:59:59.524304 12386 heartbeater.cc:464] Registering TS with master...
I20260809 00:59:59.524824 12386 heartbeater.cc:511] Master 127.4.82.190:44177 requested a full tablet report, sending...
I20260809 00:59:59.526023 11042 ts_manager.cc:194] Re-registered known tserver with Master: 964f6e9dde52455299287e03815ae164 (127.4.82.131:39315)
I20260809 00:59:59.527365 11042 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:52561
I20260809 00:59:59.536638 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.494100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53112 (local address 127.4.82.189:33311)
0809 00:59:59.508905 (+ 14805us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.508913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.508935 (+ 22us) server_negotiation.cc:407] Connection header received
0809 00:59:59.508986 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.508992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.509066 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.514657 (+ 5591us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:59.514891 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.516120 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.518940 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.520246 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.520368 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.528839 (+ 8471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:59.528848 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:59.528857 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:59.528903 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:59.532746 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.533688 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.533694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.533698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.533876 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.534402 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.534407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.534411 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.534789 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:59.535008 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.535864 (+ 856us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.536079 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14662,"thread_start_us":14480,"threads_started":1}
I20260809 00:59:59.538228 12403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.499952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47332 (local address 127.4.82.190:44177)
0809 00:59:59.509046 (+ 9094us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.509053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.509081 (+ 28us) server_negotiation.cc:407] Connection header received
0809 00:59:59.509131 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.509136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.509308 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.509422 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:59.521366 (+ 11944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.523007 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.525236 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.526919 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.527033 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.531064 (+ 4031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:59.531072 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:59.531081 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:59.531111 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:59.534054 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.535072 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.535076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.535078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.535189 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.535705 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.535712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.535716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.536171 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:59.536431 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.537533 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.537765 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":8960,"thread_start_us":9398,"threads_started":1}
I20260809 00:59:59.544374 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.504695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49976 (local address 127.4.82.188:42363)
0809 00:59:59.510616 (+ 5921us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.510622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.526466 (+ 15844us) server_negotiation.cc:407] Connection header received
0809 00:59:59.526731 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.526736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.526843 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.526917 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:59.529855 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.531128 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.533043 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.533770 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.533879 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.537615 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:59.537621 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:59.537626 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:59.537669 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:59.541328 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.542209 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.542217 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.542220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.542392 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.542763 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.542768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.542771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.543138 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:59.543440 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.543759 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.543994 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5825,"thread_start_us":20993,"threads_started":1}
I20260809 00:59:59.542857 11042 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:59:59.551502 11042 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:59:59.598690 12418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.571344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:39315 (local address 127.0.0.1:46530)
0809 00:59:59.591935 (+ 20591us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:59.591956 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:59.592156 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:59.592897 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:59.592902 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:59.592929 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:59.593410 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.593424 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.594986 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.594992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.596397 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.596408 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.597798 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.597805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.597968 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.598008 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 00:59:59.598147 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 00:59:59.598310 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20399,"thread_start_us":20251,"threads_started":1}
I20260809 00:59:59.598861 12420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.571670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46530 (local address 127.4.82.131:39315)
0809 00:59:59.591169 (+ 19499us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.591183 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.592135 (+ 952us) server_negotiation.cc:407] Connection header received
0809 00:59:59.592342 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.592351 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.592681 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.592897 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:59.593645 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.594849 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.596543 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.597589 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.597743 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.597970 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.598337 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.598522 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19265,"thread_start_us":19523,"threads_started":1}
I20260809 00:59:59.602963 12316 tablet_service.cc:1467] Processing CreateTablet for tablet b1ad586fa0734385bf0f7c8d16cbd0c1 (DEFAULT_TABLE table=test-table [id=8c96353ff2fc4db0a0bef0bef49222b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:59.604311 12316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1ad586fa0734385bf0f7c8d16cbd0c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:59.608105 12415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.570477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:42733 (local address 127.0.0.1:48044)
0809 00:59:59.585843 (+ 15366us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:59.585865 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:59.585993 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:59.600376 (+ 14383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:59.600380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:59.600406 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:59.600867 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.600883 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.602977 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.602982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.605508 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.605520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.607326 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.607332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.607487 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.607526 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 00:59:59.607615 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 00:59:59.607740 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15231,"mutex_wait_us":51,"spinlock_wait_cycles":7936,"thread_start_us":403,"threads_started":1}
I20260809 00:59:59.608119 12416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.570078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:39777 (local address 127.0.0.1:52464)
0809 00:59:59.571173 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:59.585180 (+ 14007us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:59.585451 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:59.600405 (+ 14954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:59.600409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:59.600435 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:59.600933 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.600979 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.603053 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.603057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.605506 (+ 2449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.605520 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.606884 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.606890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.607068 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.607101 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 00:59:59.607215 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 00:59:59.607379 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":965,"mutex_wait_us":254,"thread_start_us":15498,"threads_started":1}
I20260809 00:59:59.608134 12419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.571368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48044 (local address 127.4.82.130:42733)
0809 00:59:59.599735 (+ 28367us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.599743 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.599771 (+ 28us) server_negotiation.cc:407] Connection header received
0809 00:59:59.599866 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.599874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.600161 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.601112 (+ 951us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:59.601320 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.602776 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.605708 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.607086 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.607217 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.607439 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.607667 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.607840 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28197,"thread_start_us":28356,"threads_started":1}
I20260809 00:59:59.609220 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.570836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52464 (local address 127.4.82.129:39777)
0809 00:59:59.599784 (+ 28948us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.599793 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.599824 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:59:59.599916 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.599926 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.600167 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.600420 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:59.601126 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.602576 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.605728 (+ 3152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.606685 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.606831 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.607037 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.608682 (+ 1645us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.608977 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28781,"thread_start_us":28675,"threads_started":1}
I20260809 00:59:59.612118 12161 tablet_service.cc:1467] Processing CreateTablet for tablet b1ad586fa0734385bf0f7c8d16cbd0c1 (DEFAULT_TABLE table=test-table [id=8c96353ff2fc4db0a0bef0bef49222b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:59.612594 12002 tablet_service.cc:1467] Processing CreateTablet for tablet b1ad586fa0734385bf0f7c8d16cbd0c1 (DEFAULT_TABLE table=test-table [id=8c96353ff2fc4db0a0bef0bef49222b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:59:59.613234 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1ad586fa0734385bf0f7c8d16cbd0c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:59.613853 12002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1ad586fa0734385bf0f7c8d16cbd0c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:59:59.620352 12421 tablet_bootstrap.cc:492] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164: Bootstrap starting.
I20260809 00:59:59.624253 12421 tablet_bootstrap.cc:654] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:59.625506 12421 log.cc:824] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:59.626881 12423 tablet_bootstrap.cc:492] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54: Bootstrap starting.
I20260809 00:59:59.628750 12424 tablet_bootstrap.cc:492] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53: Bootstrap starting.
I20260809 00:59:59.629014 12421 tablet_bootstrap.cc:492] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164: No bootstrap required, opened a new log
I20260809 00:59:59.629223 12421 ts_tablet_manager.cc:1397] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 00:59:59.631237 12423 tablet_bootstrap.cc:654] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:59.632092 12424 tablet_bootstrap.cc:654] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53: Neither blocks nor log segments found. Creating new log.
I20260809 00:59:59.632501 12423 log.cc:824] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:59.633308 12424 log.cc:824] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53: Log is configured to *not* fsync() on all Append() calls
I20260809 00:59:59.635794 12423 tablet_bootstrap.cc:492] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54: No bootstrap required, opened a new log
I20260809 00:59:59.636068 12423 ts_tablet_manager.cc:1397] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 00:59:59.637133 12421 raft_consensus.cc:348] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } } peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } }
I20260809 00:59:59.637490 12421 raft_consensus.cc:374] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:59.637617 12421 raft_consensus.cc:729] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964f6e9dde52455299287e03815ae164, State: Initialized, Role: FOLLOWER
I20260809 00:59:59.637955 12424 tablet_bootstrap.cc:492] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53: No bootstrap required, opened a new log
I20260809 00:59:59.638208 12424 ts_tablet_manager.cc:1397] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 00:59:59.638219 12421 consensus_queue.cc:260] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } } peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } }
I20260809 00:59:59.640136 12386 heartbeater.cc:503] Master 127.4.82.190:44177 was elected leader, sending a full tablet report...
I20260809 00:59:59.640774 12421 ts_tablet_manager.cc:1428] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164: Time spent starting tablet: real 0.011s user 0.003s sys 0.008s
I20260809 00:59:59.646757 12423 raft_consensus.cc:348] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } } peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } }
I20260809 00:59:59.647229 12423 raft_consensus.cc:374] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:59.647359 12423 raft_consensus.cc:729] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c702680b924d28995faa26a770ed54, State: Initialized, Role: FOLLOWER
I20260809 00:59:59.648121 12423 consensus_queue.cc:260] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } } peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } }
I20260809 00:59:59.649346 12424 raft_consensus.cc:348] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } } peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } }
I20260809 00:59:59.649698 12424 raft_consensus.cc:374] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:59:59.649819 12424 raft_consensus.cc:729] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d14b4ca8b0140d2a29afca36f844a53, State: Initialized, Role: FOLLOWER
I20260809 00:59:59.649994 12232 heartbeater.cc:503] Master 127.4.82.190:44177 was elected leader, sending a full tablet report...
I20260809 00:59:59.650521 12424 consensus_queue.cc:260] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } } peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } }
I20260809 00:59:59.652210 12423 ts_tablet_manager.cc:1428] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
I20260809 00:59:59.653823 12424 ts_tablet_manager.cc:1428] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260809 00:59:59.673202 12428 raft_consensus.cc:482] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:59:59.673483 12428 raft_consensus.cc:504] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } } peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } }
I20260809 00:59:59.675101 12428 leader_election.cc:288] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d14b4ca8b0140d2a29afca36f844a53 (127.4.82.129:39777), 964f6e9dde52455299287e03815ae164 (127.4.82.131:39315)
I20260809 00:59:59.682142 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.675733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:44863 (local address 127.4.82.129:39777)
0809 00:59:59.675916 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.675922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.676495 (+ 573us) server_negotiation.cc:407] Connection header received
0809 00:59:59.676718 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.676722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.676778 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.676872 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:59.677551 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.678440 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.680351 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.681192 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.681368 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.681453 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.681771 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.681889 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 00:59:59.682169 12420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.675626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:35057 (local address 127.4.82.131:39315)
0809 00:59:59.675851 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.675860 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.676497 (+ 637us) server_negotiation.cc:407] Connection header received
0809 00:59:59.676738 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.676742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.676797 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.676892 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:59:59.677620 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.678347 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.680227 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.681183 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.681381 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.681456 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.681821 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.681938 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:59:59.682317 12433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.675606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:39315 (local address 127.4.82.130:35057)
0809 00:59:59.676379 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:59.676393 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:59.676594 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:59.676905 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:59.676909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:59.676931 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:59.677385 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.677394 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.678484 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.678491 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.680075 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.680087 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.681467 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.681477 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.681649 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.681671 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 00:59:59.681794 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 00:59:59.681911 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":67,"spinlock_wait_cycles":9344,"thread_start_us":529,"threads_started":1}
I20260809 00:59:59.682746 12431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.675446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:39777 (local address 127.4.82.130:44863)
0809 00:59:59.676335 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0809 00:59:59.676356 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 00:59:59.676576 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:59:59.676887 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:59:59.676891 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:59:59.676914 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:59:59.677348 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.677360 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.678576 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.678583 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.680175 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:59:59.680186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.681451 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.681457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:59:59.681612 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.681635 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 00:59:59.681740 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 00:59:59.681857 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":5,"spinlock_wait_cycles":8448,"thread_start_us":414,"threads_started":1}
I20260809 00:59:59.683326 12336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ad586fa0734385bf0f7c8d16cbd0c1" candidate_uuid: "27c702680b924d28995faa26a770ed54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964f6e9dde52455299287e03815ae164" is_pre_election: true
I20260809 00:59:59.683732 12025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ad586fa0734385bf0f7c8d16cbd0c1" candidate_uuid: "27c702680b924d28995faa26a770ed54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d14b4ca8b0140d2a29afca36f844a53" is_pre_election: true
I20260809 00:59:59.683812 12336 raft_consensus.cc:2454] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27c702680b924d28995faa26a770ed54 in term 0.
I20260809 00:59:59.684177 12025 raft_consensus.cc:2454] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27c702680b924d28995faa26a770ed54 in term 0.
I20260809 00:59:59.684512 12112 leader_election.cc:302] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27c702680b924d28995faa26a770ed54, 964f6e9dde52455299287e03815ae164; no voters:
I20260809 00:59:59.684935 12428 raft_consensus.cc:2781] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:59:59.685103 12428 raft_consensus.cc:482] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:59:59.685217 12428 raft_consensus.cc:3037] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:59.687489 12428 raft_consensus.cc:504] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } } peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } }
I20260809 00:59:59.688257 12428 leader_election.cc:288] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [CANDIDATE]: Term 1 election: Requested vote from peers 3d14b4ca8b0140d2a29afca36f844a53 (127.4.82.129:39777), 964f6e9dde52455299287e03815ae164 (127.4.82.131:39315)
I20260809 00:59:59.688650 12025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ad586fa0734385bf0f7c8d16cbd0c1" candidate_uuid: "27c702680b924d28995faa26a770ed54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d14b4ca8b0140d2a29afca36f844a53"
I20260809 00:59:59.688861 12025 raft_consensus.cc:3037] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:59.688844 12336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1ad586fa0734385bf0f7c8d16cbd0c1" candidate_uuid: "27c702680b924d28995faa26a770ed54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964f6e9dde52455299287e03815ae164"
I20260809 00:59:59.689031 12336 raft_consensus.cc:3037] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:59:59.691149 12025 raft_consensus.cc:2454] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27c702680b924d28995faa26a770ed54 in term 1.
I20260809 00:59:59.691257 12336 raft_consensus.cc:2454] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27c702680b924d28995faa26a770ed54 in term 1.
I20260809 00:59:59.691550 12113 leader_election.cc:302] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27c702680b924d28995faa26a770ed54, 3d14b4ca8b0140d2a29afca36f844a53; no voters:
I20260809 00:59:59.691789 12428 raft_consensus.cc:2781] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:59:59.692221 12428 raft_consensus.cc:686] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [term 1 LEADER]: Becoming Leader. State: Replica: 27c702680b924d28995faa26a770ed54, State: Running, Role: LEADER
I20260809 00:59:59.692816 12428 consensus_queue.cc:237] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } } peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } }
I20260809 00:59:59.698338 11042 catalog_manager.cc:5495] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27c702680b924d28995faa26a770ed54 (127.4.82.130). New cstate: current_term: 1 leader_uuid: "27c702680b924d28995faa26a770ed54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } health_report { overall_health: UNKNOWN } } }
W20260809 00:59:59.722019 12388 tablet.cc:2369] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:59:59.723778 12419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.718229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48058 (local address 127.4.82.130:42733)
0809 00:59:59.718440 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.718446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.718578 (+ 132us) server_negotiation.cc:407] Connection header received
0809 00:59:59.718818 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.718822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.718896 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.719042 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:59:59.719458 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.720220 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.721604 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.722013 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.722150 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.722657 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:59:59.723151 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:59:59.723343 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.723424 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.723508 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 00:59:59.731943 12336 raft_consensus.cc:1261] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164 [term 1 FOLLOWER]: Refusing update from remote peer 27c702680b924d28995faa26a770ed54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:59:59.731973 12025 raft_consensus.cc:1261] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53 [term 1 FOLLOWER]: Refusing update from remote peer 27c702680b924d28995faa26a770ed54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 00:59:59.732910 12077 tablet.cc:2369] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:59:59.732923 12428 consensus_queue.cc:1035] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:59.733265 12434 consensus_queue.cc:1035] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:59:59.746703 12438 mvcc.cc:205] Tried to move back new op lower bound from 7316427570092806144 to 7316427569946251264. Current Snapshot: MvccSnapshot[applied={T|T < 7316427570092806144}]
I20260809 00:59:59.748672 12440 mvcc.cc:205] Tried to move back new op lower bound from 7316427570092806144 to 7316427569946251264. Current Snapshot: MvccSnapshot[applied={T|T < 7316427570092806144}]
I20260809 00:59:59.763792 11042 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:59.764020 11042 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:59:59.770895 11042 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0bb0e225e8e46a8b608e6094288b96b: Sending DeleteTablet for 3 replicas of tablet b1ad586fa0734385bf0f7c8d16cbd0c1
I20260809 00:59:59.771889 12002 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ad586fa0734385bf0f7c8d16cbd0c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:59 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:52464
I20260809 00:59:59.772734 12447 tablet_replica.cc:330] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53: stopping tablet replica
I20260809 00:59:59.773226 12447 raft_consensus.cc:2229] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:59.773391 12161 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ad586fa0734385bf0f7c8d16cbd0c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:59 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:48044
I20260809 00:59:59.773598 12316 tablet_service.cc:1514] Processing DeleteTablet for tablet b1ad586fa0734385bf0f7c8d16cbd0c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:59:59 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:46530
I20260809 00:59:59.773952 12447 raft_consensus.cc:2258] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:59.776335 12447 ts_tablet_manager.cc:1905] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:59.785987 12447 ts_tablet_manager.cc:1918] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:59:59.786659 12449 tablet_replica.cc:330] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164: stopping tablet replica
I20260809 00:59:59.787140 12449 raft_consensus.cc:2229] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:59:59.787674 12449 raft_consensus.cc:2258] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:59.788123 12447 log.cc:1196] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/b1ad586fa0734385bf0f7c8d16cbd0c1
I20260809 00:59:59.788506 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 11009
I20260809 00:59:59.788738 12447 ts_tablet_manager.cc:1939] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 3d14b4ca8b0140d2a29afca36f844a53: Deleting consensus metadata
I20260809 00:59:59.789865 11075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 00:59:59.792878 12449 ts_tablet_manager.cc:1905] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:59.796823 12448 tablet_replica.cc:330] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54: stopping tablet replica
I20260809 00:59:59.797370 12448 raft_consensus.cc:2229] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:59:59.798599 12448 raft_consensus.cc:2258] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:59:59.798987 12449 ts_tablet_manager.cc:1918] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:59:59.799172 12449 log.cc:1196] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/b1ad586fa0734385bf0f7c8d16cbd0c1
I20260809 00:59:59.799794 12449 ts_tablet_manager.cc:1939] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 964f6e9dde52455299287e03815ae164: Deleting consensus metadata
I20260809 00:59:59.801270 12448 ts_tablet_manager.cc:1905] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:59:59.810400 12448 ts_tablet_manager.cc:1918] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:59:59.810604 12448 log.cc:1196] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/b1ad586fa0734385bf0f7c8d16cbd0c1
I20260809 00:59:59.812511 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.775408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53124 (local address 127.4.82.189:33311)
0809 00:59:59.775630 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.775637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.786636 (+ 10999us) server_negotiation.cc:407] Connection header received
0809 00:59:59.786867 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.786873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.786954 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.787057 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:59.790946 (+ 3889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.791945 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.797557 (+ 5612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.798240 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.798337 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.802382 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:59.802392 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:59.802399 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:59.802444 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:59.805876 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.806726 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.806737 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.806741 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.806857 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.807243 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.807249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.807252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.807646 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:59.807894 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.811574 (+ 3680us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.811884 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 00:59:59.815814 12448 ts_tablet_manager.cc:1939] T b1ad586fa0734385bf0f7c8d16cbd0c1 P 27c702680b924d28995faa26a770ed54: Deleting consensus metadata
I20260809 00:59:59.825932 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:59:59.787330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49978 (local address 127.4.82.188:42363)
0809 00:59:59.787503 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 00:59:59.787509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:59:59.812227 (+ 24718us) server_negotiation.cc:407] Connection header received
0809 00:59:59.812477 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:59:59.812484 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:59:59.812576 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:59:59.812666 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:59:59.814560 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.815378 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.817192 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:59:59.817744 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:59:59.817820 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:59:59.820565 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:59:59.820571 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:59:59.820576 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:59:59.820617 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:59:59.823248 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.823941 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.823948 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.823951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.824089 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:59:59.824456 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:59:59.824462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:59:59.824465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:59:59.824738 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:59:59.825011 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 00:59:59.825326 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 00:59:59.825534 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:59:59.982560 11030 catalog_manager.cc:4841] TS 3d14b4ca8b0140d2a29afca36f844a53 (127.4.82.129:39777): tablet b1ad586fa0734385bf0f7c8d16cbd0c1 (table test-table [id=8c96353ff2fc4db0a0bef0bef49222b6]) successfully deleted
W20260809 00:59:59.982892 11030 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.82.129:39777, user_credentials={real_user=slave}} blocked reactor thread for 192266us
I20260809 00:59:59.983335 11029 catalog_manager.cc:4841] TS 964f6e9dde52455299287e03815ae164 (127.4.82.131:39315): tablet b1ad586fa0734385bf0f7c8d16cbd0c1 (table test-table [id=8c96353ff2fc4db0a0bef0bef49222b6]) successfully deleted
W20260809 00:59:59.983528 11029 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.82.131:39315, user_credentials={real_user=slave}} blocked reactor thread for 181905us
I20260809 00:59:59.983815 11029 catalog_manager.cc:4841] TS 27c702680b924d28995faa26a770ed54 (127.4.82.130:42733): tablet b1ad586fa0734385bf0f7c8d16cbd0c1 (table test-table [id=8c96353ff2fc4db0a0bef0bef49222b6]) successfully deleted
I20260809 00:59:59.987193 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 11009
I20260809 01:00:00.041978 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:00.024273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53126 (local address 127.4.82.189:33311)
0809 01:00:00.024502 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:00.024512 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:00.025459 (+ 947us) server_negotiation.cc:407] Connection header received
0809 01:00:00.025746 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:00.025753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:00.025838 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:00.025913 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:00.028347 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:00.029467 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:00.031442 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:00.032057 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:00.032146 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:00.035812 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:00.035819 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:00.035824 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:00.035869 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:00.039179 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:00.039828 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:00.039838 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:00.039840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:00.039955 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:00.040344 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:00.040349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:00.040354 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:00.040746 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:00.041012 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:00.041349 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 01:00:00.041581 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 01:00:00.043778 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:00.025195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49992 (local address 127.4.82.188:42363)
0809 01:00:00.025464 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:00.025473 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:00.026392 (+ 919us) server_negotiation.cc:407] Connection header received
0809 01:00:00.026706 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:00.026715 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:00.026814 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:00.026901 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:00.030109 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:00.031568 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:00.033469 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:00.033889 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:00.034021 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:00.036993 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:00.036999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:00.037004 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:00.037044 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:00.040405 (+ 3361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:00.041163 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:00.041171 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:00.041176 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:00.041333 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:00.041772 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:00.041778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:00.041782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:00.042147 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:00.042385 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:00.043394 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0809 01:00:00.043549 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 01:00:00.754657 12461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:00.753248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.130:49685)
0809 01:00:00.754278 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:00.754424 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":881,"thread_start_us":781,"threads_started":1}
I20260809 01:00:00.754880 12460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:00.753107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.131:47177)
0809 01:00:00.754497 (+ 1390us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:00.754648 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1271,"thread_start_us":1140,"threads_started":1}
W20260809 01:00:00.755142 12232 heartbeater.cc:644] Failed to heartbeat to 127.4.82.190:44177 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
W20260809 01:00:00.755277 12386 heartbeater.cc:644] Failed to heartbeat to 127.4.82.190:44177 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
I20260809 01:00:00.758327 12462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:00.757470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.129:59061)
0809 01:00:00.757944 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:00.758112 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":234,"threads_started":1}
W20260809 01:00:00.758734 12075 heartbeater.cc:644] Failed to heartbeat to 127.4.82.190:44177 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
I20260809 01:00:01.270792 12463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c0bb0e225e8e46a8b608e6094288b96b)
I20260809 01:00:01.271031 12463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } }
I20260809 01:00:01.272325 12465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c0bb0e225e8e46a8b608e6094288b96b)
I20260809 01:00:01.272451 12463 leader_election.cc:288] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c0bb0e225e8e46a8b608e6094288b96b (127.4.82.190:44177), 1afdefeb53264bc984c30489b073cfde (127.4.82.189:33311)
I20260809 01:00:01.272666 12465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } }
I20260809 01:00:01.273650 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.272500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:47766)
0809 01:00:01.273121 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.273366 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":339,"threads_started":1}
W20260809 01:00:01.273963 11864 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
I20260809 01:00:01.277850 12465 leader_election.cc:288] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c0bb0e225e8e46a8b608e6094288b96b (127.4.82.190:44177), 43b38d7e56b6483885029e8887324a97 (127.4.82.188:42363)
I20260809 01:00:01.279654 12471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.274397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:47768)
0809 01:00:01.279001 (+ 4604us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.279203 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4471,"threads_started":1}
I20260809 01:00:01.280081 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.279654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:47784)
0809 01:00:01.279842 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.279913 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260809 01:00:01.280160 11670 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
W20260809 01:00:01.280351 11864 leader_election.cc:334] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c0bb0e225e8e46a8b608e6094288b96b (127.4.82.190:44177): Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
I20260809 01:00:01.282850 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.273443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53134 (local address 127.4.82.189:33311)
0809 01:00:01.274095 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:01.274102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:01.274123 (+ 21us) server_negotiation.cc:407] Connection header received
0809 01:00:01.274190 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:01.274194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:01.274265 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:01.275239 (+ 974us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:01.275550 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.277155 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.280484 (+ 3329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.281856 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.282057 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.282164 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:01.282417 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 01:00:01.282508 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":323,"threads_started":1}
I20260809 01:00:01.282902 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.272664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:33311 (local address 127.0.0.1:53134)
0809 01:00:01.273390 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.273407 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:01.273531 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:01.274456 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:01.274464 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:01.274502 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:01.274977 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.274993 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.277374 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.277383 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.280135 (+ 2752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.280155 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.282076 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.282082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.282255 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.282295 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 01:00:01.282388 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 01:00:01.282520 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":456,"threads_started":1}
I20260809 01:00:01.283398 12471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.282889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:47786)
0809 01:00:01.283112 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.283214 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260809 01:00:01.283636 11670 leader_election.cc:334] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c0bb0e225e8e46a8b608e6094288b96b (127.4.82.190:44177): Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
I20260809 01:00:01.283771 11697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43b38d7e56b6483885029e8887324a97" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1afdefeb53264bc984c30489b073cfde" is_pre_election: true
I20260809 01:00:01.284096 11697 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43b38d7e56b6483885029e8887324a97 in term 1.
I20260809 01:00:01.284605 11863 leader_election.cc:302] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1afdefeb53264bc984c30489b073cfde, 43b38d7e56b6483885029e8887324a97; no voters: c0bb0e225e8e46a8b608e6094288b96b
I20260809 01:00:01.285028 12463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 01:00:01.285167 12463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c0bb0e225e8e46a8b608e6094288b96b)
I20260809 01:00:01.285298 12463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:00:01.286893 12470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.274446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42363 (local address 127.0.0.1:50000)
0809 01:00:01.279684 (+ 5238us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.279704 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:01.279824 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:01.280506 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:01.280511 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:01.280545 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:01.281092 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.281107 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.282691 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.282707 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.284616 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.284633 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.286107 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.286114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.286288 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.286310 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 01:00:01.286432 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 01:00:01.286553 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5079,"thread_start_us":4273,"threads_started":1}
I20260809 01:00:01.286928 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.275001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50000 (local address 127.4.82.188:42363)
0809 01:00:01.278555 (+ 3554us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:01.278563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:01.279843 (+ 1280us) server_negotiation.cc:407] Connection header received
0809 01:00:01.279988 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:01.279994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:01.280079 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:01.280195 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:01.281378 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.282554 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.284801 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.285903 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.286064 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.286197 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:01.286457 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 01:00:01.286590 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3384,"thread_start_us":3187,"threads_started":1}
I20260809 01:00:01.287503 11891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1afdefeb53264bc984c30489b073cfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "43b38d7e56b6483885029e8887324a97" is_pre_election: true
I20260809 01:00:01.287881 12463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } }
I20260809 01:00:01.288290 11891 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1afdefeb53264bc984c30489b073cfde in current term 2: Already voted for candidate 43b38d7e56b6483885029e8887324a97 in this term.
I20260809 01:00:01.288872 12463 leader_election.cc:288] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [CANDIDATE]: Term 2 election: Requested vote from peers c0bb0e225e8e46a8b608e6094288b96b (127.4.82.190:44177), 1afdefeb53264bc984c30489b073cfde (127.4.82.189:33311)
I20260809 01:00:01.288961 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.288590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:47802)
0809 01:00:01.288732 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.288813 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 01:00:01.288919 11669 leader_election.cc:302] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1afdefeb53264bc984c30489b073cfde; no voters: 43b38d7e56b6483885029e8887324a97, c0bb0e225e8e46a8b608e6094288b96b
I20260809 01:00:01.289351 11697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43b38d7e56b6483885029e8887324a97" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1afdefeb53264bc984c30489b073cfde"
I20260809 01:00:01.289464 12465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:00:01.289937 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.289512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:47818)
0809 01:00:01.289704 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.289799 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260809 01:00:01.290102 11864 leader_election.cc:334] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c0bb0e225e8e46a8b608e6094288b96b (127.4.82.190:44177): Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
I20260809 01:00:01.291668 12465 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 01:00:01.293740 11697 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43b38d7e56b6483885029e8887324a97 in term 2.
I20260809 01:00:01.294298 11863 leader_election.cc:302] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1afdefeb53264bc984c30489b073cfde, 43b38d7e56b6483885029e8887324a97; no voters: c0bb0e225e8e46a8b608e6094288b96b
I20260809 01:00:01.294633 12463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 01:00:01.295136 12463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [term 2 LEADER]: Becoming Leader. State: Replica: 43b38d7e56b6483885029e8887324a97, State: Running, Role: LEADER
I20260809 01:00:01.295537 12463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } }
I20260809 01:00:01.297473 12474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43b38d7e56b6483885029e8887324a97. Latest consensus state: current_term: 2 leader_uuid: "43b38d7e56b6483885029e8887324a97" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } } }
I20260809 01:00:01.297905 12474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:01.298579 12476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:00:01.301956 12476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:00:01.302798 12476 catalog_manager.cc:1259] Loaded cluster ID: 91c290b0219847f5a904518c3570ed7b
I20260809 01:00:01.302889 12476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:00:01.304140 12476 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:00:01.305008 12476 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: Loaded TSK: 0
I20260809 01:00:01.305809 12476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:00:01.316732 11879 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:00:01.323523 11879 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:00:01.327387 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.326967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:47828)
0809 01:00:01.327129 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.327205 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 01:00:01.327455 11697 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [term 2 FOLLOWER]: Refusing update from remote peer 43b38d7e56b6483885029e8887324a97: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 01:00:01.328213 12463 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 01:00:01.328420 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.327932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:47838)
0809 01:00:01.328105 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.328209 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 01:00:01.328631 11864 consensus_peers.cc:487] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 -> Peer c0bb0e225e8e46a8b608e6094288b96b (127.4.82.190:44177): Couldn't send request to peer c0bb0e225e8e46a8b608e6094288b96b. Status: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 01:00:01.330720 12465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43b38d7e56b6483885029e8887324a97. Latest consensus state: current_term: 2 leader_uuid: "43b38d7e56b6483885029e8887324a97" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } } }
I20260809 01:00:01.331029 12465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:00:01.333232 12465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "43b38d7e56b6483885029e8887324a97" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } } }
I20260809 01:00:01.333496 12465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1afdefeb53264bc984c30489b073cfde [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:00:01.333572 12463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "43b38d7e56b6483885029e8887324a97" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c0bb0e225e8e46a8b608e6094288b96b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44177 } } peers { permanent_uuid: "1afdefeb53264bc984c30489b073cfde" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 33311 } attrs { promote: false } } peers { permanent_uuid: "43b38d7e56b6483885029e8887324a97" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42363 } attrs { promote: false } } }
I20260809 01:00:01.333880 12463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:01.350734 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.341201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:39777 (local address 127.0.0.1:52466)
0809 01:00:01.341356 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.341369 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:01.341447 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:01.342382 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:01.342387 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:01.342415 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:01.342775 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.342791 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.344597 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.344605 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.347188 (+ 2583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.347205 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.349637 (+ 2432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.349647 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.349961 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.350018 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 01:00:01.350100 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 01:00:01.350291 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 01:00:01.352519 12484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.341669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52466 (local address 127.4.82.129:39777)
0809 01:00:01.342120 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:01.342125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:01.342138 (+ 13us) server_negotiation.cc:407] Connection header received
0809 01:00:01.342202 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:01.342206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:01.342257 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:01.342365 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:01.342963 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.344398 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.347416 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.349319 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.351727 (+ 2408us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.351832 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:01.351889 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 01:00:01.352093 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":249,"threads_started":1}
I20260809 01:00:01.353572 12002 tablet_service.cc:1467] Processing CreateTablet for tablet 694b5c39d02c402fab96f4e7bb52e77a (DEFAULT_TABLE table=test-table [id=e9bc681c41a8459b91250ac6ef1be99c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:01.354177 12002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 694b5c39d02c402fab96f4e7bb52e77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:01.360865 12487 tablet_bootstrap.cc:492] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53: Bootstrap starting.
I20260809 01:00:01.363808 12487 tablet_bootstrap.cc:654] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:01.364826 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.340560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:39315 (local address 127.0.0.1:46546)
0809 01:00:01.340806 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.340824 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:01.340941 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:01.357527 (+ 16586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:01.357535 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:01.357572 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:01.357801 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.357814 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.359281 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.359314 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.361822 (+ 2508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.361843 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.364036 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.364041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.364283 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.364319 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 01:00:01.364385 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 01:00:01.364528 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260809 01:00:01.365792 12483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.340692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46546 (local address 127.4.82.131:39315)
0809 01:00:01.357201 (+ 16509us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:01.357207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:01.357226 (+ 19us) server_negotiation.cc:407] Connection header received
0809 01:00:01.357291 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:01.357296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:01.357373 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:01.357474 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:01.358020 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.359135 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.362021 (+ 2886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.363762 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.365210 (+ 1448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.365328 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:01.365377 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 01:00:01.365520 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16363,"thread_start_us":16875,"threads_started":1}
I20260809 01:00:01.367249 12316 tablet_service.cc:1467] Processing CreateTablet for tablet 694b5c39d02c402fab96f4e7bb52e77a (DEFAULT_TABLE table=test-table [id=e9bc681c41a8459b91250ac6ef1be99c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:01.367702 12487 tablet_bootstrap.cc:492] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53: No bootstrap required, opened a new log
I20260809 01:00:01.367692 12316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 694b5c39d02c402fab96f4e7bb52e77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:01.367828 12487 ts_tablet_manager.cc:1397] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 01:00:01.368580 12487 raft_consensus.cc:348] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } } peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } }
I20260809 01:00:01.368866 12487 raft_consensus.cc:374] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:01.369009 12487 raft_consensus.cc:729] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d14b4ca8b0140d2a29afca36f844a53, State: Initialized, Role: FOLLOWER
I20260809 01:00:01.369300 12487 consensus_queue.cc:260] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } } peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } }
I20260809 01:00:01.370273 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.342905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:42733 (local address 127.0.0.1:48070)
0809 01:00:01.350965 (+ 8060us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.350982 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:01.351076 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:01.364657 (+ 13581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:01.364662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:01.364689 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:01.364867 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.364881 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.366424 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.366432 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.368081 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.368096 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.369476 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.369481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.369705 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.369764 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 01:00:01.369870 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 01:00:01.369991 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7983,"thread_start_us":23549,"threads_started":1}
I20260809 01:00:01.370282 12486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.343071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48070 (local address 127.4.82.130:42733)
0809 01:00:01.364238 (+ 21167us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:01.364245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:01.364262 (+ 17us) server_negotiation.cc:407] Connection header received
0809 01:00:01.364331 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:01.364336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:01.364418 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:01.365023 (+ 605us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:01.365214 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.366237 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.368260 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.369326 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.369455 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.369547 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:01.369894 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 01:00:01.370016 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21028,"thread_start_us":21406,"threads_started":1}
I20260809 01:00:01.370559 12487 ts_tablet_manager.cc:1428] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 01:00:01.371403 12161 tablet_service.cc:1467] Processing CreateTablet for tablet 694b5c39d02c402fab96f4e7bb52e77a (DEFAULT_TABLE table=test-table [id=e9bc681c41a8459b91250ac6ef1be99c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:01.371896 12161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 694b5c39d02c402fab96f4e7bb52e77a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:01.373310 12074 heartbeater.cc:503] Master 127.4.82.188:42363 was elected leader, sending a full tablet report...
I20260809 01:00:01.380587 12491 tablet_bootstrap.cc:492] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164: Bootstrap starting.
I20260809 01:00:01.382030 12491 tablet_bootstrap.cc:654] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:01.383916 12491 tablet_bootstrap.cc:492] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164: No bootstrap required, opened a new log
I20260809 01:00:01.384069 12491 ts_tablet_manager.cc:1397] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 01:00:01.385124 12491 raft_consensus.cc:348] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } } peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } }
I20260809 01:00:01.385479 12491 raft_consensus.cc:374] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:01.385604 12491 raft_consensus.cc:729] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964f6e9dde52455299287e03815ae164, State: Initialized, Role: FOLLOWER
I20260809 01:00:01.385845 12493 tablet_bootstrap.cc:492] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54: Bootstrap starting.
I20260809 01:00:01.385910 12491 consensus_queue.cc:260] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } } peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } }
I20260809 01:00:01.387280 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.371072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.129:41039)
0809 01:00:01.386954 (+ 15882us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.387078 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15757,"thread_start_us":15617,"threads_started":1}
I20260809 01:00:01.387914 12491 ts_tablet_manager.cc:1428] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260809 01:00:01.388078 12493 tablet_bootstrap.cc:654] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:01.389191 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.388246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.131:45273)
0809 01:00:01.388812 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.388941 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"threads_started":1}
I20260809 01:00:01.390369 12385 heartbeater.cc:503] Master 127.4.82.188:42363 was elected leader, sending a full tablet report...
I20260809 01:00:01.391348 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.390616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.131:42323)
0809 01:00:01.391087 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.391180 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377}
I20260809 01:00:01.392375 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.391984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.129:48897)
0809 01:00:01.392178 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.392266 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 01:00:01.393229 12493 tablet_bootstrap.cc:492] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54: No bootstrap required, opened a new log
I20260809 01:00:01.393337 12493 ts_tablet_manager.cc:1397] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 01:00:01.393883 12493 raft_consensus.cc:348] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } } peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } }
I20260809 01:00:01.394042 12493 raft_consensus.cc:374] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:01.394147 12493 raft_consensus.cc:729] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c702680b924d28995faa26a770ed54, State: Initialized, Role: FOLLOWER
I20260809 01:00:01.394397 12493 consensus_queue.cc:260] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } } peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } }
I20260809 01:00:01.395215 12493 ts_tablet_manager.cc:1428] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 01:00:01.396626 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.395527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.130:37207)
0809 01:00:01.396170 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.396338 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":788,"threads_started":1}
I20260809 01:00:01.397053 12231 heartbeater.cc:503] Master 127.4.82.188:42363 was elected leader, sending a full tablet report...
I20260809 01:00:01.398283 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.397845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.130:39743)
0809 01:00:01.398045 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.398143 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260809 01:00:01.584924 12234 tablet.cc:2369] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:00:01.621068 12494 raft_consensus.cc:482] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:00:01.621338 12494 raft_consensus.cc:504] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } } peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } }
I20260809 01:00:01.623062 12494 leader_election.cc:288] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d14b4ca8b0140d2a29afca36f844a53 (127.4.82.129:39777), 27c702680b924d28995faa26a770ed54 (127.4.82.130:42733)
I20260809 01:00:01.628988 12486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.623558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:46781 (local address 127.4.82.130:42733)
0809 01:00:01.623810 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:01.623819 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:01.623885 (+ 66us) server_negotiation.cc:407] Connection header received
0809 01:00:01.624161 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:01.624165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:01.624227 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:01.624338 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:01.624802 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.625523 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.627196 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.628036 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.628232 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.628307 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:01.628593 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 01:00:01.628729 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 01:00:01.629012 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.623436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:42733 (local address 127.4.82.131:46781)
0809 01:00:01.623766 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.623794 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:01.623977 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:01.624374 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:01.624379 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:01.624425 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:01.624654 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.624662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.625672 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.625679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.627032 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.627043 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.628243 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.628249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.628414 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.628439 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 01:00:01.628573 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 01:00:01.628717 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205}
I20260809 01:00:01.629272 12484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.623533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:35653 (local address 127.4.82.129:39777)
0809 01:00:01.623759 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:01.623766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:01.624324 (+ 558us) server_negotiation.cc:407] Connection header received
0809 01:00:01.624523 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:01.624528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:01.624601 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:01.624702 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:01.625158 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.625863 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.627576 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.628334 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.628510 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.628602 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:01.628872 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 01:00:01.629000 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 01:00:01.629827 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.623445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:39777 (local address 127.4.82.131:35653)
0809 01:00:01.624218 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.624231 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:01.624368 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:01.624720 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:01.624724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:01.624747 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:01.624997 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.625005 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.626006 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.626012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.627420 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:01.627433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.628538 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.628555 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:01.628721 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.628742 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 01:00:01.628845 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 01:00:01.628968 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":38,"thread_start_us":442,"threads_started":1}
I20260809 01:00:01.629906 12182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "694b5c39d02c402fab96f4e7bb52e77a" candidate_uuid: "964f6e9dde52455299287e03815ae164" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27c702680b924d28995faa26a770ed54" is_pre_election: true
I20260809 01:00:01.630406 12182 raft_consensus.cc:2454] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964f6e9dde52455299287e03815ae164 in term 0.
I20260809 01:00:01.630561 12025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "694b5c39d02c402fab96f4e7bb52e77a" candidate_uuid: "964f6e9dde52455299287e03815ae164" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d14b4ca8b0140d2a29afca36f844a53" is_pre_election: true
I20260809 01:00:01.630852 12025 raft_consensus.cc:2454] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 964f6e9dde52455299287e03815ae164 in term 0.
I20260809 01:00:01.631042 12263 leader_election.cc:302] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27c702680b924d28995faa26a770ed54, 964f6e9dde52455299287e03815ae164; no voters:
I20260809 01:00:01.631464 12494 raft_consensus.cc:2781] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:00:01.631597 12494 raft_consensus.cc:482] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:00:01.631696 12494 raft_consensus.cc:3037] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:01.633832 12494 raft_consensus.cc:504] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } } peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } }
I20260809 01:00:01.634407 12494 leader_election.cc:288] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [CANDIDATE]: Term 1 election: Requested vote from peers 3d14b4ca8b0140d2a29afca36f844a53 (127.4.82.129:39777), 27c702680b924d28995faa26a770ed54 (127.4.82.130:42733)
I20260809 01:00:01.634804 12025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "694b5c39d02c402fab96f4e7bb52e77a" candidate_uuid: "964f6e9dde52455299287e03815ae164" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d14b4ca8b0140d2a29afca36f844a53"
I20260809 01:00:01.634897 12182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "694b5c39d02c402fab96f4e7bb52e77a" candidate_uuid: "964f6e9dde52455299287e03815ae164" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27c702680b924d28995faa26a770ed54"
I20260809 01:00:01.635011 12025 raft_consensus.cc:3037] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:01.635126 12182 raft_consensus.cc:3037] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:01.637104 12025 raft_consensus.cc:2454] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964f6e9dde52455299287e03815ae164 in term 1.
I20260809 01:00:01.637487 12265 leader_election.cc:302] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d14b4ca8b0140d2a29afca36f844a53, 964f6e9dde52455299287e03815ae164; no voters:
I20260809 01:00:01.637701 12182 raft_consensus.cc:2454] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 964f6e9dde52455299287e03815ae164 in term 1.
I20260809 01:00:01.637780 12494 raft_consensus.cc:2781] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:00:01.638293 12494 raft_consensus.cc:686] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [term 1 LEADER]: Becoming Leader. State: Replica: 964f6e9dde52455299287e03815ae164, State: Running, Role: LEADER
I20260809 01:00:01.638924 12494 consensus_queue.cc:237] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } } peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } }
I20260809 01:00:01.639362 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.638905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.131:41239)
0809 01:00:01.639054 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.639183 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 01:00:01.642554 11879 catalog_manager.cc:5495] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 reported cstate change: term changed from 0 to 1, leader changed from <none> to 964f6e9dde52455299287e03815ae164 (127.4.82.131). New cstate: current_term: 1 leader_uuid: "964f6e9dde52455299287e03815ae164" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964f6e9dde52455299287e03815ae164" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 } health_report { overall_health: UNKNOWN } } }
I20260809 01:00:01.720397 12483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.715029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46550 (local address 127.4.82.131:39315)
0809 01:00:01.715244 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:01.715251 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:01.715346 (+ 95us) server_negotiation.cc:407] Connection header received
0809 01:00:01.715610 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:01.715614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:01.715707 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:01.715861 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:00:01.716237 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.716924 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.718214 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.718642 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.718780 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.719186 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:00:01.719771 (+ 585us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:00:01.719930 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:01.720045 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 01:00:01.720127 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 01:00:01.724282 12025 raft_consensus.cc:1261] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53 [term 1 FOLLOWER]: Refusing update from remote peer 964f6e9dde52455299287e03815ae164: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:01.724728 12182 raft_consensus.cc:1261] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54 [term 1 FOLLOWER]: Refusing update from remote peer 964f6e9dde52455299287e03815ae164: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:01.725033 12500 consensus_queue.cc:1035] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d14b4ca8b0140d2a29afca36f844a53" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 39777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:00:01.725514 12494 consensus_queue.cc:1035] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27c702680b924d28995faa26a770ed54" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:00:01.728020 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.727475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.130:37259)
0809 01:00:01.727699 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.727804 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 01:00:01.728293 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.727789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.129:37329)
0809 01:00:01.728015 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.728135 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260809 01:00:01.734910 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.734258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.130:42523)
0809 01:00:01.734561 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.734693 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260809 01:00:01.738394 12503 mvcc.cc:205] Tried to move back new op lower bound from 7316427578256048128 to 7316427577917071360. Current Snapshot: MvccSnapshot[applied={T|T < 7316427578256048128}]
I20260809 01:00:01.741063 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.740496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.131:33727)
0809 01:00:01.740785 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.740891 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
I20260809 01:00:01.743943 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.743473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.129:46569)
0809 01:00:01.743704 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.743788 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260809 01:00:01.744146 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.743677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.4.82.131:39675)
0809 01:00:01.743876 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.743967 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 01:00:01.764257 11880 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:00:01.764540 11880 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:00:01.770509 11880 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43b38d7e56b6483885029e8887324a97: Sending DeleteTablet for 3 replicas of tablet 694b5c39d02c402fab96f4e7bb52e77a
I20260809 01:00:01.771692 12316 tablet_service.cc:1514] Processing DeleteTablet for tablet 694b5c39d02c402fab96f4e7bb52e77a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:01 UTC) from {username='oryx', principal='oryx/127.4.82.188@KRBTEST.COM'} at 127.0.0.1:46546
I20260809 01:00:01.771788 12002 tablet_service.cc:1514] Processing DeleteTablet for tablet 694b5c39d02c402fab96f4e7bb52e77a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:01 UTC) from {username='oryx', principal='oryx/127.4.82.188@KRBTEST.COM'} at 127.0.0.1:52466
I20260809 01:00:01.772475 12513 tablet_replica.cc:330] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164: stopping tablet replica
I20260809 01:00:01.772814 12161 tablet_service.cc:1514] Processing DeleteTablet for tablet 694b5c39d02c402fab96f4e7bb52e77a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:01 UTC) from {username='oryx', principal='oryx/127.4.82.188@KRBTEST.COM'} at 127.0.0.1:48070
I20260809 01:00:01.772944 12513 raft_consensus.cc:2229] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:00:01.773635 12513 raft_consensus.cc:2258] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:01.775197 12513 ts_tablet_manager.cc:1905] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:01.775357 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 11850
I20260809 01:00:01.784545 12514 tablet_replica.cc:330] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53: stopping tablet replica
I20260809 01:00:01.784757 12514 raft_consensus.cc:2229] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:00:01.785043 12513 ts_tablet_manager.cc:1918] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:00:01.785224 12513 log.cc:1196] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/694b5c39d02c402fab96f4e7bb52e77a
I20260809 01:00:01.785264 12514 raft_consensus.cc:2258] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:01.785923 12513 ts_tablet_manager.cc:1939] T 694b5c39d02c402fab96f4e7bb52e77a P 964f6e9dde52455299287e03815ae164: Deleting consensus metadata
I20260809 01:00:01.786904 11865 catalog_manager.cc:4841] TS 964f6e9dde52455299287e03815ae164 (127.4.82.131:39315): tablet 694b5c39d02c402fab96f4e7bb52e77a (table test-table [id=e9bc681c41a8459b91250ac6ef1be99c]) successfully deleted
I20260809 01:00:01.788170 12514 ts_tablet_manager.cc:1905] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:01.788645 12515 tablet_replica.cc:330] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54: stopping tablet replica
I20260809 01:00:01.788862 12515 raft_consensus.cc:2229] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:00:01.789173 12515 raft_consensus.cc:2258] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:01.789984 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.789492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:47846)
0809 01:00:01.789708 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.789803 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260809 01:00:01.791182 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.790749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:47848)
0809 01:00:01.790946 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:01.791028 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 01:00:01.794581 12514 ts_tablet_manager.cc:1918] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:00:01.794734 12514 log.cc:1196] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/694b5c39d02c402fab96f4e7bb52e77a
I20260809 01:00:01.795243 12514 ts_tablet_manager.cc:1939] T 694b5c39d02c402fab96f4e7bb52e77a P 3d14b4ca8b0140d2a29afca36f844a53: Deleting consensus metadata
I20260809 01:00:01.796480 11866 catalog_manager.cc:4841] TS 3d14b4ca8b0140d2a29afca36f844a53 (127.4.82.129:39777): tablet 694b5c39d02c402fab96f4e7bb52e77a (table test-table [id=e9bc681c41a8459b91250ac6ef1be99c]) successfully deleted
I20260809 01:00:01.800160 12515 ts_tablet_manager.cc:1905] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:01.806903 12515 ts_tablet_manager.cc:1918] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:00:01.807089 12515 log.cc:1196] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/694b5c39d02c402fab96f4e7bb52e77a
I20260809 01:00:01.807606 12515 ts_tablet_manager.cc:1939] T 694b5c39d02c402fab96f4e7bb52e77a P 27c702680b924d28995faa26a770ed54: Deleting consensus metadata
I20260809 01:00:01.808727 11865 catalog_manager.cc:4841] TS 27c702680b924d28995faa26a770ed54 (127.4.82.130:42733): tablet 694b5c39d02c402fab96f4e7bb52e77a (table test-table [id=e9bc681c41a8459b91250ac6ef1be99c]) successfully deleted
I20260809 01:00:01.809499 12484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.776697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52472 (local address 127.4.82.129:39777)
0809 01:00:01.776868 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:01.776875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:01.788035 (+ 11160us) server_negotiation.cc:407] Connection header received
0809 01:00:01.788240 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:01.788247 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:01.788330 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:01.788429 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:01.791383 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.792630 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.794420 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.795133 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.795279 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.803092 (+ 7813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:01.803107 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:01.803123 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:01.803162 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:01.806360 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:01.807030 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:01.807041 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:01.807044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:01.807223 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:01.807648 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:01.807653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:01.807657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:01.808042 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:01.808324 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:01.809001 (+ 677us) server_negotiation.cc:299] Negotiation successful
0809 01:00:01.809265 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 01:00:01.810078 12069 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:01.940953 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 11850
I20260809 01:00:01.959434 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 12085
I20260809 01:00:01.975116 12486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:01.960255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.4.82.130:42733)
0809 01:00:01.960446 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:01.960451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:01.960488 (+ 37us) server_negotiation.cc:407] Connection header received
0809 01:00:01.960690 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:01.960695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:01.960761 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:01.960844 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:01.962550 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.963257 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.965247 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:01.965633 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:01.965728 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:01.969151 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:01.969165 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:01.969180 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:01.969224 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:01.972391 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:01.973216 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:01.973223 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:01.973225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:01.973343 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:01.973730 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:01.973733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:01.973734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:01.974065 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:01.974292 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:01.974663 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 01:00:01.974828 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 01:00:01.975857 12226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:02.100337 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 12085
I20260809 01:00:02.125813 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 12244
I20260809 01:00:02.142257 12483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:02.126648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.4.82.131:39315)
0809 01:00:02.126847 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:02.126851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:02.126951 (+ 100us) server_negotiation.cc:407] Connection header received
0809 01:00:02.127227 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:02.127231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:02.127384 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:02.127480 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:02.129525 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:02.130296 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:02.132125 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:02.132554 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:02.132655 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:02.135947 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:02.135963 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:02.135978 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:02.136014 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:02.139184 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:02.139999 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:02.140032 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:02.140035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:02.140205 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:02.140635 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:02.140639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:02.140641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:02.141030 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:02.141378 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:02.141630 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 01:00:02.141878 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 01:00:02.142976 12380 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:02.277796 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 12244
I20260809 01:00:02.303781 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 11534
I20260809 01:00:02.304982 11719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:02.316783 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:02.316051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:47854)
0809 01:00:02.316523 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:02.316612 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":271,"threads_started":1}
I20260809 01:00:02.318120 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:02.317788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:44177 (local address 127.0.0.1:47864)
0809 01:00:02.317934 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:02.318003 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:44177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 01:00:02.422451 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 11534
I20260809 01:00:02.445734 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 11647
I20260809 01:00:02.446758 11911 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:02.569137 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 11647
2026-08-09T01:00:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9703 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:00:02 dist-test-slave-07c2 krb5kdc[12526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:00:02 dist-test-slave-07c2 krb5kdc[12526](info): set up 2 sockets
Aug 09 01:00:02 dist-test-slave-07c2 krb5kdc[12526](info): commencing operation
krb5kdc: starting...
W20260809 01:00:03.715677 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:00:03 dist-test-slave-07c2 krb5kdc[12526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:00:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:00:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.190.keytab.
Entry for principal oryx/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.190.keytab.
I20260809 01:00:03.942863 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:43655
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.190.keytab
--principal=oryx/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:43655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:04.257959 12539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:04.258229 12539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:04.258318 12539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:04.265144 12539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:00:04.265237 12539 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 01:00:04.265278 12539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:04.265334 12539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:04.265378 12539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:00:04.265439 12539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:00:04.274693 12539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:43655
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.190.keytab
--principal=oryx/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:43655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:04.275921 12539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:04.277557 12539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:04.288862 12544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:04.288887 12547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:04.288935 12545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:04.290302 12539 server_base.cc:1034] running on GCE node
I20260809 01:00:04.291085 12539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:04.292477 12539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:04.293756 12539 hybrid_clock.cc:648] HybridClock initialized: now 1786237204293700 us; error 47 us; skew 500 ppm
I20260809 01:00:04.294103 12539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:04 dist-test-slave-07c2 krb5kdc[12526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237204, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:04.297206 12539 init.cc:377] Logged in from keytab as oryx/127.4.82.190@KRBTEST.COM (short username oryx)
I20260809 01:00:04.298671 12539 webserver.cc:460] Webserver started at http://127.4.82.190:38787/ using document root <none> and password file <none>
I20260809 01:00:04.299410 12539 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:04.299549 12539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:04.299876 12539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:04.302729 12539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "ded9735aa1764e249ba71df292682ec4"
format_stamp: "Formatted at 2026-08-09 01:00:04 on dist-test-slave-07c2"
server_key: "d69ce4fdd9027d96b5d7e61db92a9a97"
server_key_iv: "cd3c6d728d2afcf0dc2c838c1438700a"
server_key_version: "encryptionkey@0"
I20260809 01:00:04.303598 12539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "ded9735aa1764e249ba71df292682ec4"
format_stamp: "Formatted at 2026-08-09 01:00:04 on dist-test-slave-07c2"
server_key: "d69ce4fdd9027d96b5d7e61db92a9a97"
server_key_iv: "cd3c6d728d2afcf0dc2c838c1438700a"
server_key_version: "encryptionkey@0"
I20260809 01:00:04.308476 12539 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 01:00:04.311721 12555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:04.312917 12539 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 01:00:04.313113 12539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "ded9735aa1764e249ba71df292682ec4"
format_stamp: "Formatted at 2026-08-09 01:00:04 on dist-test-slave-07c2"
server_key: "d69ce4fdd9027d96b5d7e61db92a9a97"
server_key_iv: "cd3c6d728d2afcf0dc2c838c1438700a"
server_key_version: "encryptionkey@0"
I20260809 01:00:04.313329 12539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:04.329603 12539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:04.333314 12539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:04.333593 12539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:04.354395 12539 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:43655
I20260809 01:00:04.354424 12621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:43655 every 8 connection(s)
I20260809 01:00:04.355696 12539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:00:04.359109 12622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:04.364688 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 12539
I20260809 01:00:04.364909 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:00:04.365768 4426 external_mini_cluster.cc:1428] Setting key fcb6ced7f32857bc9ffdcc379300b0bd
I20260809 01:00:04.369872 12622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4: Bootstrap starting.
I20260809 01:00:04.373407 12622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:04.374433 12622 log.cc:824] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4: Log is configured to *not* fsync() on all Append() calls
I20260809 01:00:04.377307 12622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4: No bootstrap required, opened a new log
Aug 09 01:00:04 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 01:00:04.387521 12622 raft_consensus.cc:348] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded9735aa1764e249ba71df292682ec4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 43655 } }
I20260809 01:00:04.387964 12622 raft_consensus.cc:374] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:04.388072 12622 raft_consensus.cc:729] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ded9735aa1764e249ba71df292682ec4, State: Initialized, Role: FOLLOWER
I20260809 01:00:04.388727 12622 consensus_queue.cc:260] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded9735aa1764e249ba71df292682ec4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 43655 } }
I20260809 01:00:04.389034 12622 raft_consensus.cc:388] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:00:04.389171 12622 raft_consensus.cc:482] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:00:04.389354 12622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:04.389678 12625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:04.367863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39574 (local address 127.4.82.190:43655)
0809 01:00:04.368692 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:04.368703 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:04.368742 (+ 39us) server_negotiation.cc:407] Connection header received
0809 01:00:04.370447 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:04.370467 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:04.370843 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:04.371143 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:04.373005 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:04.374307 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:04.375895 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:04.376518 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:04.376677 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:04.381337 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:04.381346 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:04.381363 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:04.381403 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:04.385161 (+ 3758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:04.385897 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:04.385906 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:04.385914 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:04.386105 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:04.386472 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:04.386482 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:04.386485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:04.387141 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:04.387520 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:04.387869 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 01:00:04.388282 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":399,"threads_started":1}
I20260809 01:00:04.391862 12622 raft_consensus.cc:504] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded9735aa1764e249ba71df292682ec4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 43655 } }
I20260809 01:00:04.392318 12622 leader_election.cc:302] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ded9735aa1764e249ba71df292682ec4; no voters:
I20260809 01:00:04.392832 12622 leader_election.cc:288] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:00:04.392943 12627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:00:04.393759 12627 raft_consensus.cc:686] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [term 1 LEADER]: Becoming Leader. State: Replica: ded9735aa1764e249ba71df292682ec4, State: Running, Role: LEADER
I20260809 01:00:04.394291 12627 consensus_queue.cc:237] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded9735aa1764e249ba71df292682ec4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 43655 } }
I20260809 01:00:04.394840 12622 sys_catalog.cc:564] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:00:04.397181 12629 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ded9735aa1764e249ba71df292682ec4. Latest consensus state: current_term: 1 leader_uuid: "ded9735aa1764e249ba71df292682ec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded9735aa1764e249ba71df292682ec4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 43655 } } }
I20260809 01:00:04.397643 12629 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:04.397668 12628 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ded9735aa1764e249ba71df292682ec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded9735aa1764e249ba71df292682ec4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 43655 } } }
I20260809 01:00:04.398670 12628 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:04.399474 12637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:00:04.403594 12637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:00:04.411597 12637 catalog_manager.cc:1347] Generated new cluster ID: d9314a3d5b9a4d6a837254e7aee54bb8
I20260809 01:00:04.411702 12637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:00:04.440100 12637 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:00:04.441903 12637 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:00:04.453177 12637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4: Generated new TSK 0
I20260809 01:00:04.454064 12637 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab.
Entry for principal oryx/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab.
I20260809 01:00:04.520771 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:43655
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab
--principal=oryx/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:04.844223 12649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:04.844484 12649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:04.844637 12649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:04.851717 12649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:04.851809 12649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:04.851972 12649 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 01:00:04.852093 12649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:00:04.861562 12649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab
--principal=oryx/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.82.190:43655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:04.862748 12649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:04.864073 12649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:04.876487 12654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:04.876566 12658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:04.876531 12655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:04.877637 12649 server_base.cc:1034] running on GCE node
I20260809 01:00:04.878209 12649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:04.879058 12649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:04.880293 12649 hybrid_clock.cc:648] HybridClock initialized: now 1786237204880218 us; error 56 us; skew 500 ppm
I20260809 01:00:04.880578 12649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:04 dist-test-slave-07c2 krb5kdc[12526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237204, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:04.883589 12649 init.cc:377] Logged in from keytab as oryx/127.4.82.129@KRBTEST.COM (short username oryx)
I20260809 01:00:04.884891 12649 webserver.cc:460] Webserver started at http://127.4.82.129:44545/ using document root <none> and password file <none>
I20260809 01:00:04.885547 12649 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:04.885682 12649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:04.885991 12649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:04.888903 12649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "021fcab1a6da4afbbc78776ce563549c"
format_stamp: "Formatted at 2026-08-09 01:00:04 on dist-test-slave-07c2"
server_key: "d4c4da6f73f4074417e2f9fba5725b83"
server_key_iv: "94d0746223a211358b555335cd9eb1ef"
server_key_version: "encryptionkey@0"
I20260809 01:00:04.889689 12649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "021fcab1a6da4afbbc78776ce563549c"
format_stamp: "Formatted at 2026-08-09 01:00:04 on dist-test-slave-07c2"
server_key: "d4c4da6f73f4074417e2f9fba5725b83"
server_key_iv: "94d0746223a211358b555335cd9eb1ef"
server_key_version: "encryptionkey@0"
I20260809 01:00:04.894701 12649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 01:00:04.897944 12665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:04.899264 12649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 01:00:04.899488 12649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "021fcab1a6da4afbbc78776ce563549c"
format_stamp: "Formatted at 2026-08-09 01:00:04 on dist-test-slave-07c2"
server_key: "d4c4da6f73f4074417e2f9fba5725b83"
server_key_iv: "94d0746223a211358b555335cd9eb1ef"
server_key_version: "encryptionkey@0"
I20260809 01:00:04.899701 12649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:04.926157 12649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:04.929818 12649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:04.930074 12649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:04.932616 12649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:04.932723 12649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:04.932883 12649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:04.932986 12649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 01:00:04 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237204, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 01:00:04.956598 12673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:04.932869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:34725 (local address 127.4.82.190:43655)
0809 01:00:04.933472 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:04.933479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:04.934609 (+ 1130us) server_negotiation.cc:407] Connection header received
0809 01:00:04.936600 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:04.936607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:04.936703 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:04.936817 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:04.938848 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:04.940225 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:04.943600 (+ 3375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:04.944197 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:04.944340 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:04.949415 (+ 5075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:04.949420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:04.949424 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:04.949455 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:04.952639 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:04.953794 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:04.953799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:04.953802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:04.953976 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:04.954746 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:04.954754 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:04.954759 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:04.955114 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:04.955272 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:04.956014 (+ 742us) server_negotiation.cc:299] Negotiation successful
0809 01:00:04.956234 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":349,"threads_started":1}
I20260809 01:00:04.957458 12675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:04.933475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:43655 (local address 127.4.82.129:34725)
0809 01:00:04.934417 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:04.934461 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:04.936340 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:04.937039 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:04.937046 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:04.937787 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:04.938637 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:04.938665 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:04.940474 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:04.940481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:04.943412 (+ 2931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:04.943432 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:04.944474 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:04.944484 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:04.944838 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:04.945665 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:04.945687 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:04.949104 (+ 3417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:04.952839 (+ 3735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:04.952849 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:04.952863 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:04.953635 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:04.954148 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:04.954155 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:04.954161 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:04.954487 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:04.955256 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:04.955263 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:04.955669 (+ 406us) client_negotiation.cc:769] Sending connection context
0809 01:00:04.955917 (+ 248us) client_negotiation.cc:240] Negotiation successful
0809 01:00:04.956390 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":413,"threads_started":1}
I20260809 01:00:04.982137 12673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:04.967064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:55799 (local address 127.4.82.190:43655)
0809 01:00:04.967308 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:04.967316 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:04.967844 (+ 528us) server_negotiation.cc:407] Connection header received
0809 01:00:04.967978 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:04.967986 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:04.968085 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:04.968210 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:04.969047 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:04.970444 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:04.972071 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:04.972646 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:04.972795 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:04.975031 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:04.975040 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:04.975047 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:04.975096 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:04.978732 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:04.979718 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:04.979726 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:04.979732 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:04.979901 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:04.980426 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:04.980434 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:04.980438 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:04.980831 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:04.981031 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:04.981532 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 01:00:04.981761 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 01:00:04.982247 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:04.967083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:43655 (local address 127.4.82.129:55799)
0809 01:00:04.967735 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:04.967760 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:04.967864 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:04.968231 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:04.968234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:04.968632 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:04.968912 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:04.968920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:04.970605 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:04.970612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:04.971856 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:04.971882 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:04.972871 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:04.972886 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:04.973046 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:04.973781 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:04.973809 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:04.974828 (+ 1019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:04.978934 (+ 4106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:04.978941 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:04.978947 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:04.979577 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:04.980057 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:04.980063 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:04.980067 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:04.980255 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:04.980987 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:04.980995 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:04.981271 (+ 276us) client_negotiation.cc:769] Sending connection context
0809 01:00:04.981432 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 01:00:04.981762 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":375,"threads_started":1}
I20260809 01:00:04.987398 12649 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:43233
I20260809 01:00:04.987432 12792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:43233 every 8 connection(s)
I20260809 01:00:04.988502 12649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:00:04.990321 12793 heartbeater.cc:347] Connected to a master server at 127.4.82.190:43655
I20260809 01:00:04.990546 12793 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:04.991130 12793 heartbeater.cc:511] Master 127.4.82.190:43655 requested a full tablet report, sending...
I20260809 01:00:04.992938 12573 ts_manager.cc:194] Registered new tserver with Master: 021fcab1a6da4afbbc78776ce563549c (127.4.82.129:43233)
I20260809 01:00:04.994429 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 12649
I20260809 01:00:04.994499 12573 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:34725
I20260809 01:00:04.994684 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:00:04.995635 4426 external_mini_cluster.cc:1428] Setting key feeef04559de2d6e3dc8d3d18f5871a9
WARNING: no policy specified for oryx/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab.
Entry for principal oryx/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab.
I20260809 01:00:05.045635 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:43655
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab
--principal=oryx/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:05.368892 12797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:05.369216 12797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:05.369352 12797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:05.379477 12797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:05.379572 12797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:05.379664 12797 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 01:00:05.379741 12797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:00:05.389048 12797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab
--principal=oryx/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.82.190:43655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:05.390172 12797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:05.391359 12797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:05.404228 12803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:05.404304 12806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:05.404345 12804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:05.405674 12797 server_base.cc:1034] running on GCE node
I20260809 01:00:05.406241 12797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:05.407044 12797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:05.408288 12797 hybrid_clock.cc:648] HybridClock initialized: now 1786237205408206 us; error 59 us; skew 500 ppm
I20260809 01:00:05.408571 12797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:05 dist-test-slave-07c2 krb5kdc[12526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237205, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:05.411724 12797 init.cc:377] Logged in from keytab as oryx/127.4.82.130@KRBTEST.COM (short username oryx)
I20260809 01:00:05.413093 12797 webserver.cc:460] Webserver started at http://127.4.82.130:40157/ using document root <none> and password file <none>
I20260809 01:00:05.413746 12797 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:05.413856 12797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:05.414197 12797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:05.417088 12797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "c6fd0b18618e4ecda64db5c7acb15b5d"
format_stamp: "Formatted at 2026-08-09 01:00:05 on dist-test-slave-07c2"
server_key: "46019ca90abaaffd191c71372fd80643"
server_key_iv: "6e62c3bed50c8722da8ec6487b8254e1"
server_key_version: "encryptionkey@0"
I20260809 01:00:05.417958 12797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "c6fd0b18618e4ecda64db5c7acb15b5d"
format_stamp: "Formatted at 2026-08-09 01:00:05 on dist-test-slave-07c2"
server_key: "46019ca90abaaffd191c71372fd80643"
server_key_iv: "6e62c3bed50c8722da8ec6487b8254e1"
server_key_version: "encryptionkey@0"
I20260809 01:00:05.422677 12797 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 01:00:05.425899 12815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:05.427076 12797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 01:00:05.427238 12797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "c6fd0b18618e4ecda64db5c7acb15b5d"
format_stamp: "Formatted at 2026-08-09 01:00:05 on dist-test-slave-07c2"
server_key: "46019ca90abaaffd191c71372fd80643"
server_key_iv: "6e62c3bed50c8722da8ec6487b8254e1"
server_key_version: "encryptionkey@0"
I20260809 01:00:05.427528 12797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:05.440100 12797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:05.444052 12797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:05.444329 12797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:05.446851 12797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:05.447007 12797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 01:00:05.447196 12797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:05.447387 12797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 01:00:05 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237205, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 01:00:05.470404 12673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:05.447187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:43027 (local address 127.4.82.190:43655)
0809 01:00:05.447407 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:05.447412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:05.448760 (+ 1348us) server_negotiation.cc:407] Connection header received
0809 01:00:05.451036 (+ 2276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:05.451045 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:05.451138 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:05.451251 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:05.453359 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:05.454362 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:05.456884 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:05.457548 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:05.457685 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:05.461964 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:05.461968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:05.461971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:05.462003 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:05.465404 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:05.466370 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:05.466378 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:05.466384 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:05.466546 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:05.467067 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:05.467072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:05.467076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:05.467521 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:05.467682 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:05.469896 (+ 2214us) server_negotiation.cc:299] Negotiation successful
0809 01:00:05.470166 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 01:00:05.470790 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:05.447822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:43655 (local address 127.4.82.130:43027)
0809 01:00:05.448568 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:05.448628 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:05.450712 (+ 2084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:05.451532 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:05.451546 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:05.452316 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:05.453156 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:05.453187 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:05.454529 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:05.454534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:05.456744 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:05.456760 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:05.457823 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:05.457832 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:05.458097 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:05.458859 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:05.458889 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:05.461738 (+ 2849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:05.465536 (+ 3798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:05.465544 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:05.465560 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:05.466207 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:05.466696 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:05.466703 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:05.466711 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:05.466935 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:05.467753 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:05.467758 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:05.468281 (+ 523us) client_negotiation.cc:769] Sending connection context
0809 01:00:05.468574 (+ 293us) client_negotiation.cc:240] Negotiation successful
0809 01:00:05.469070 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":341,"threads_started":1}
I20260809 01:00:05.491247 12673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:05.476294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:54653 (local address 127.4.82.190:43655)
0809 01:00:05.476455 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:05.476459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:05.477395 (+ 936us) server_negotiation.cc:407] Connection header received
0809 01:00:05.477566 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:05.477571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:05.477653 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:05.477763 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:05.478942 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:05.479827 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:05.481097 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:05.481723 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:05.481845 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:05.484993 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:05.484997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:05.485000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:05.485038 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:05.488238 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:05.489218 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:05.489221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:05.489224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:05.489344 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:05.489777 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:05.489779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:05.489781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:05.490070 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:05.490250 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:05.490678 (+ 428us) server_negotiation.cc:299] Negotiation successful
0809 01:00:05.490892 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 01:00:05.491374 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:05.476658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:43655 (local address 127.4.82.130:54653)
0809 01:00:05.477290 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:05.477312 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:05.477457 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:05.477788 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:05.477792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:05.478305 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:05.478721 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:05.478733 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:05.479966 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:05.479971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:05.480943 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:05.480954 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:05.482041 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:05.482052 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:05.482482 (+ 430us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:05.483396 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:05.483430 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:05.484762 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:05.488396 (+ 3634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:05.488404 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:05.488411 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:05.489090 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:05.489460 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:05.489467 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:05.489473 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:05.489667 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:05.490228 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:05.490233 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:05.490431 (+ 198us) client_negotiation.cc:769] Sending connection context
0809 01:00:05.490573 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 01:00:05.490894 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":393,"threads_started":1}
I20260809 01:00:05.505676 12797 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:42037
I20260809 01:00:05.505707 12940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:42037 every 8 connection(s)
I20260809 01:00:05.506925 12797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:00:05.509016 12941 heartbeater.cc:347] Connected to a master server at 127.4.82.190:43655
I20260809 01:00:05.509057 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 12797
I20260809 01:00:05.509227 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:00:05.509256 12941 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:05.509789 4426 external_mini_cluster.cc:1428] Setting key 6c2bb683209085d733365b1d05f22c69
I20260809 01:00:05.510051 12941 heartbeater.cc:511] Master 127.4.82.190:43655 requested a full tablet report, sending...
I20260809 01:00:05.511696 12571 ts_manager.cc:194] Registered new tserver with Master: c6fd0b18618e4ecda64db5c7acb15b5d (127.4.82.130:42037)
I20260809 01:00:05.513084 12571 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:43027
WARNING: no policy specified for oryx/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab.
Entry for principal oryx/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab.
I20260809 01:00:05.564551 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:43655
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab
--principal=oryx/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:05.891525 12945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:05.891777 12945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:05.891961 12945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:05.899152 12945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:05.899273 12945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:05.899436 12945 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 01:00:05.899561 12945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:00:05.909143 12945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab
--principal=oryx/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.82.190:43655
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:05.910320 12945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:05.911751 12945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:05.924456 12950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:05.924515 12953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:05.924547 12951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:05.925367 12945 server_base.cc:1034] running on GCE node
I20260809 01:00:05.925963 12945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:05.927136 12945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:05.928494 12945 hybrid_clock.cc:648] HybridClock initialized: now 1786237205928402 us; error 58 us; skew 500 ppm
I20260809 01:00:05.928857 12945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:05 dist-test-slave-07c2 krb5kdc[12526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237205, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:05.932104 12945 init.cc:377] Logged in from keytab as oryx/127.4.82.131@KRBTEST.COM (short username oryx)
I20260809 01:00:05.933442 12945 webserver.cc:460] Webserver started at http://127.4.82.131:37813/ using document root <none> and password file <none>
I20260809 01:00:05.934111 12945 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:05.934243 12945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:05.934563 12945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:05.937386 12945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "93e6c924eebd4ca99eec02ff46236160"
format_stamp: "Formatted at 2026-08-09 01:00:05 on dist-test-slave-07c2"
server_key: "4f678b2343633cc627b60ae3c357fa19"
server_key_iv: "cd85b4387a79c71919f5cb34a5293d6f"
server_key_version: "encryptionkey@0"
I20260809 01:00:05.938184 12945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "93e6c924eebd4ca99eec02ff46236160"
format_stamp: "Formatted at 2026-08-09 01:00:05 on dist-test-slave-07c2"
server_key: "4f678b2343633cc627b60ae3c357fa19"
server_key_iv: "cd85b4387a79c71919f5cb34a5293d6f"
server_key_version: "encryptionkey@0"
I20260809 01:00:05.943522 12945 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 01:00:05.946770 12961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:05.948072 12945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:00:05.948247 12945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "93e6c924eebd4ca99eec02ff46236160"
format_stamp: "Formatted at 2026-08-09 01:00:05 on dist-test-slave-07c2"
server_key: "4f678b2343633cc627b60ae3c357fa19"
server_key_iv: "cd85b4387a79c71919f5cb34a5293d6f"
server_key_version: "encryptionkey@0"
I20260809 01:00:05.948485 12945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:05.984529 12945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:05.989328 12945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:05.989631 12945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:05.993569 12945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:05.993733 12945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 01:00:05.993968 12945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:05.994181 12945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:05.997524 12793 heartbeater.cc:503] Master 127.4.82.190:43655 was elected leader, sending a full tablet report...
Aug 09 01:00:06 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237205, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 01:00:06.019753 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:05.993596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:52155 (local address 127.4.82.190:43655)
0809 01:00:05.994102 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:05.994111 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:05.995355 (+ 1244us) server_negotiation.cc:407] Connection header received
0809 01:00:05.999438 (+ 4083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:05.999446 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:05.999552 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:05.999650 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:06.002813 (+ 3163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.003962 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.006392 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.006912 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.007056 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.012766 (+ 5710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:06.012771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:06.012774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:06.012808 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:06.016593 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:06.017328 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:06.017333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:06.017337 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:06.017511 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:06.017951 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:06.017957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:06.017961 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:06.018246 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:06.018407 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.019091 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.019335 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":244,"threads_started":1}
I20260809 01:00:06.020740 12972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:05.994201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:43655 (local address 127.4.82.131:52155)
0809 01:00:05.995166 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:05.995211 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:05.997996 (+ 2785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:06.000273 (+ 2277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:06.000286 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:06.001362 (+ 1076us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:06.002523 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.002555 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.004150 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.004154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.006152 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.006168 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.007363 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.007373 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.007568 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.008499 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:06.008525 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:06.012509 (+ 3984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:06.016733 (+ 4224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:06.016742 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:06.016759 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:06.017200 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:06.017635 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:06.017641 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:06.017645 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:06.017840 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:06.018375 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:06.018382 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:06.018721 (+ 339us) client_negotiation.cc:769] Sending connection context
0809 01:00:06.018982 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 01:00:06.019578 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":470,"threads_started":1}
I20260809 01:00:06.043299 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.026992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:53421 (local address 127.4.82.190:43655)
0809 01:00:06.027139 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.027143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.028165 (+ 1022us) server_negotiation.cc:407] Connection header received
0809 01:00:06.028393 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.028398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.028479 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.028692 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:06.030363 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.031325 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.033475 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.033913 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.034052 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.036236 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:06.036241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:06.036244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:06.036272 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:06.040146 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:06.040968 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:06.040976 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:06.040982 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:06.041170 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:06.041630 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:06.041637 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:06.041641 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:06.042000 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:06.042172 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.042663 (+ 491us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.042887 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 01:00:06.043403 13031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.027108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:43655 (local address 127.4.82.131:53421)
0809 01:00:06.028042 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:06.028062 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:06.028246 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:06.028890 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:06.028896 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:06.029645 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:06.030139 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.030158 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.031463 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.031469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.033322 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.033340 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.034119 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.034125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.034291 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.034954 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:06.034970 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:06.035963 (+ 993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:06.040284 (+ 4321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:06.040288 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:06.040290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:06.040828 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:06.041310 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:06.041318 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:06.041323 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:06.041508 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:06.042234 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:06.042238 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:06.042406 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 01:00:06.042560 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 01:00:06.042887 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":551,"threads_started":1}
I20260809 01:00:06.053130 12945 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:34163
I20260809 01:00:06.053154 13088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:34163 every 8 connection(s)
I20260809 01:00:06.054277 12945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:00:06.056375 13089 heartbeater.cc:347] Connected to a master server at 127.4.82.190:43655
I20260809 01:00:06.056658 13089 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:06.057384 13089 heartbeater.cc:511] Master 127.4.82.190:43655 requested a full tablet report, sending...
I20260809 01:00:06.058972 12571 ts_manager.cc:194] Registered new tserver with Master: 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131:34163)
I20260809 01:00:06.059847 12571 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:52155
I20260809 01:00:06.061244 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 12945
I20260809 01:00:06.061432 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:00:06.061882 4426 external_mini_cluster.cc:1428] Setting key 654da109694916ec0d9c20c9e97dd033
I20260809 01:00:06.064679 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 01:00:06 dist-test-slave-07c2 krb5kdc[12526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237206, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 01:00:06 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:06 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 01:00:06.121829 12970 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:06.111128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39578 (local address 127.4.82.190:43655)
0809 01:00:06.111371 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.111378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.111550 (+ 172us) server_negotiation.cc:407] Connection header received
0809 01:00:06.111819 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.111823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.111888 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.111998 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:06.113379 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.114105 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.115419 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.115861 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.116001 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.121556 (+ 5555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39578: BlockingRecv error: recv got EOF from 127.0.0.1:39578 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 01:00:06 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237206, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 01:00:06.137702 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.124849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39592 (local address 127.4.82.190:43655)
0809 01:00:06.125056 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.125064 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.125353 (+ 289us) server_negotiation.cc:407] Connection header received
0809 01:00:06.125548 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.125552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.125616 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.125721 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:06.126786 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.127551 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.128855 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.129273 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.129408 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.133051 (+ 3643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:06.133056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:06.133059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:06.133089 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:06.135623 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:06.136207 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:06.136210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:06.136213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:06.136318 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:06.136621 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:06.136624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:06.136626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:06.136860 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:06.137038 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.137330 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.137481 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 01:00:06.141376 12571 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:00:06.148245 12571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:00:06.170238 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.159436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55952 (local address 127.4.82.130:42037)
0809 01:00:06.160135 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.160145 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.161551 (+ 1406us) server_negotiation.cc:407] Connection header received
0809 01:00:06.161670 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.161681 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.161965 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.162193 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:06.162988 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.164573 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.167409 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.168799 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.168912 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.169116 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.169772 (+ 656us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.169952 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":383,"threads_started":1}
I20260809 01:00:06.171329 13112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.159222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:42037 (local address 127.0.0.1:55952)
0809 01:00:06.161070 (+ 1848us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:06.161124 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:06.161335 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:06.162231 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:06.162242 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:06.162281 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:06.162709 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.162723 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.164736 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.164740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.167191 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.167206 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.169310 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.169317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.169502 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.169555 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 01:00:06.170744 (+ 1189us) client_negotiation.cc:240] Negotiation successful
0809 01:00:06.170924 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1519,"thread_start_us":3645,"threads_started":1}
I20260809 01:00:06.171907 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.160369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42224 (local address 127.4.82.129:43233)
0809 01:00:06.162806 (+ 2437us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.162816 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.163468 (+ 652us) server_negotiation.cc:407] Connection header received
0809 01:00:06.163577 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.163584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.163780 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.163947 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:06.164785 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.166317 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.168784 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.170226 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.170359 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.170591 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.171426 (+ 835us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.171600 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2282,"thread_start_us":2166,"threads_started":1}
I20260809 01:00:06.171746 13110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.160189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:43233 (local address 127.0.0.1:42224)
0809 01:00:06.163168 (+ 2979us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:06.163191 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:06.163343 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:06.164158 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:06.164162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:06.164186 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:06.164599 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.164612 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.166473 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.166477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.168574 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.168606 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.170741 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.170747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.170919 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.170949 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 01:00:06.171036 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 01:00:06.171159 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2838,"thread_start_us":367,"threads_started":1}
I20260809 01:00:06.176278 12720 tablet_service.cc:1467] Processing CreateTablet for tablet 298e607ac23b49e39afe51324f16f5ec (DEFAULT_TABLE table=test-table [id=e1822129436d4c3b888fecff26352b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:06.177064 12868 tablet_service.cc:1467] Processing CreateTablet for tablet 298e607ac23b49e39afe51324f16f5ec (DEFAULT_TABLE table=test-table [id=e1822129436d4c3b888fecff26352b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:06.177732 12720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 298e607ac23b49e39afe51324f16f5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:06.178325 12868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 298e607ac23b49e39afe51324f16f5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:06.180452 13112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.163271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:34163 (local address 127.0.0.1:52148)
0809 01:00:06.171992 (+ 8721us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:06.172014 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:06.172121 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:06.174027 (+ 1906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:06.174031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:06.174050 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:06.174206 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.174215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.176130 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.176135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.177930 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.177946 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.179747 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.179754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.179929 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.179960 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 01:00:06.180060 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 01:00:06.180199 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8579,"thread_start_us":10563,"threads_started":1}
I20260809 01:00:06.180622 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.163685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52148 (local address 127.4.82.131:34163)
0809 01:00:06.172943 (+ 9258us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.172950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.172981 (+ 31us) server_negotiation.cc:407] Connection header received
0809 01:00:06.173069 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.173076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.173292 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.173475 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:06.174426 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.175968 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.178120 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.179562 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.179710 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.179917 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.180117 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.180275 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9070,"thread_start_us":8980,"threads_started":1}
I20260809 01:00:06.184111 13015 tablet_service.cc:1467] Processing CreateTablet for tablet 298e607ac23b49e39afe51324f16f5ec (DEFAULT_TABLE table=test-table [id=e1822129436d4c3b888fecff26352b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:06.185362 13015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 298e607ac23b49e39afe51324f16f5ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:06.189393 13117 tablet_bootstrap.cc:492] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d: Bootstrap starting.
I20260809 01:00:06.191952 13118 tablet_bootstrap.cc:492] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c: Bootstrap starting.
I20260809 01:00:06.192893 13117 tablet_bootstrap.cc:654] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:06.194168 13117 log.cc:824] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d: Log is configured to *not* fsync() on all Append() calls
I20260809 01:00:06.195417 13118 tablet_bootstrap.cc:654] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:06.196666 13118 log.cc:824] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c: Log is configured to *not* fsync() on all Append() calls
I20260809 01:00:06.197299 13117 tablet_bootstrap.cc:492] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d: No bootstrap required, opened a new log
I20260809 01:00:06.197619 13117 ts_tablet_manager.cc:1397] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260809 01:00:06.199713 13121 tablet_bootstrap.cc:492] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160: Bootstrap starting.
I20260809 01:00:06.199777 13118 tablet_bootstrap.cc:492] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c: No bootstrap required, opened a new log
I20260809 01:00:06.200171 13118 ts_tablet_manager.cc:1397] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260809 01:00:06.202665 13121 tablet_bootstrap.cc:654] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:06.204136 13121 log.cc:824] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160: Log is configured to *not* fsync() on all Append() calls
I20260809 01:00:06.206521 13121 tablet_bootstrap.cc:492] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160: No bootstrap required, opened a new log
I20260809 01:00:06.206779 13121 ts_tablet_manager.cc:1397] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260809 01:00:06.208627 13117 raft_consensus.cc:348] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } }
I20260809 01:00:06.209051 13117 raft_consensus.cc:374] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:06.209151 13117 raft_consensus.cc:729] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6fd0b18618e4ecda64db5c7acb15b5d, State: Initialized, Role: FOLLOWER
I20260809 01:00:06.209715 13117 consensus_queue.cc:260] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } }
I20260809 01:00:06.210716 12941 heartbeater.cc:503] Master 127.4.82.190:43655 was elected leader, sending a full tablet report...
I20260809 01:00:06.211419 13117 ts_tablet_manager.cc:1428] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 01:00:06.211725 13118 raft_consensus.cc:348] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } }
I20260809 01:00:06.212141 13118 raft_consensus.cc:374] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:06.212275 13118 raft_consensus.cc:729] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021fcab1a6da4afbbc78776ce563549c, State: Initialized, Role: FOLLOWER
I20260809 01:00:06.212895 13118 consensus_queue.cc:260] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } }
I20260809 01:00:06.214599 13118 ts_tablet_manager.cc:1428] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 01:00:06.215381 13121 raft_consensus.cc:348] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } }
I20260809 01:00:06.215788 13121 raft_consensus.cc:374] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:06.215957 13121 raft_consensus.cc:729] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e6c924eebd4ca99eec02ff46236160, State: Initialized, Role: FOLLOWER
I20260809 01:00:06.216593 13121 consensus_queue.cc:260] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } }
I20260809 01:00:06.217718 13089 heartbeater.cc:503] Master 127.4.82.190:43655 was elected leader, sending a full tablet report...
I20260809 01:00:06.218410 13121 ts_tablet_manager.cc:1428] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
W20260809 01:00:06.240825 12794 tablet.cc:2369] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 01:00:06.259024 12942 tablet.cc:2369] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 01:00:06.305935 13090 tablet.cc:2369] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:00:06.454124 13123 raft_consensus.cc:482] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:00:06.454455 13123 raft_consensus.cc:504] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } }
I20260809 01:00:06.456259 13123 leader_election.cc:288] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131:34163), 021fcab1a6da4afbbc78776ce563549c (127.4.82.129:43233)
I20260809 01:00:06.465127 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.456780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:58241 (local address 127.4.82.131:34163)
0809 01:00:06.456975 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.456983 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.457657 (+ 674us) server_negotiation.cc:407] Connection header received
0809 01:00:06.457880 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.457885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.457966 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.458077 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:06.458925 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.460052 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.462886 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.464130 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.464310 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.464383 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.464725 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.464877 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 01:00:06.465173 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.456506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:34163 (local address 127.4.82.130:58241)
0809 01:00:06.457537 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:06.457557 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:06.457732 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:06.458105 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:06.458110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:06.458150 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:06.458765 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.458778 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.460190 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.460196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.462736 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.462745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.464353 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.464360 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.464532 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.464575 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 01:00:06.464679 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 01:00:06.464814 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"mutex_wait_us":92,"spinlock_wait_cycles":4480,"thread_start_us":666,"threads_started":1}
I20260809 01:00:06.465261 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.456839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:46169 (local address 127.4.82.129:43233)
0809 01:00:06.457079 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.457088 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.457714 (+ 626us) server_negotiation.cc:407] Connection header received
0809 01:00:06.457879 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.457885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.457966 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.458077 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:06.458904 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.460055 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.462886 (+ 2831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.464180 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.464315 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.464383 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.464906 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.465034 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 01:00:06.465932 13126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.456627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:43233 (local address 127.4.82.130:46169)
0809 01:00:06.457608 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:06.457625 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:06.457742 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:06.458105 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:06.458110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:06.458142 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:06.458649 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.458668 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.460192 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.460196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.462726 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.462741 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.464538 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.464543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.464720 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.464755 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 01:00:06.465428 (+ 673us) client_negotiation.cc:240] Negotiation successful
0809 01:00:06.465561 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"mutex_wait_us":151,"spinlock_wait_cycles":24320,"thread_start_us":498,"threads_started":1}
I20260809 01:00:06.466264 13040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298e607ac23b49e39afe51324f16f5ec" candidate_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93e6c924eebd4ca99eec02ff46236160" is_pre_election: true
I20260809 01:00:06.466778 13040 raft_consensus.cc:2454] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6fd0b18618e4ecda64db5c7acb15b5d in term 0.
I20260809 01:00:06.466957 12741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298e607ac23b49e39afe51324f16f5ec" candidate_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "021fcab1a6da4afbbc78776ce563549c" is_pre_election: true
I20260809 01:00:06.467425 12819 leader_election.cc:302] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93e6c924eebd4ca99eec02ff46236160, c6fd0b18618e4ecda64db5c7acb15b5d; no voters:
I20260809 01:00:06.467489 12741 raft_consensus.cc:2454] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6fd0b18618e4ecda64db5c7acb15b5d in term 0.
I20260809 01:00:06.467814 13123 raft_consensus.cc:2781] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:00:06.467978 13123 raft_consensus.cc:482] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:00:06.468098 13123 raft_consensus.cc:3037] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:06.470227 13123 raft_consensus.cc:504] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } }
I20260809 01:00:06.470845 13123 leader_election.cc:288] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [CANDIDATE]: Term 1 election: Requested vote from peers 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131:34163), 021fcab1a6da4afbbc78776ce563549c (127.4.82.129:43233)
I20260809 01:00:06.471199 13040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298e607ac23b49e39afe51324f16f5ec" candidate_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93e6c924eebd4ca99eec02ff46236160"
I20260809 01:00:06.471371 12741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "298e607ac23b49e39afe51324f16f5ec" candidate_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "021fcab1a6da4afbbc78776ce563549c"
I20260809 01:00:06.471421 13040 raft_consensus.cc:3037] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:06.471545 12741 raft_consensus.cc:3037] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:06.473866 13040 raft_consensus.cc:2454] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6fd0b18618e4ecda64db5c7acb15b5d in term 1.
I20260809 01:00:06.474138 12741 raft_consensus.cc:2454] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6fd0b18618e4ecda64db5c7acb15b5d in term 1.
I20260809 01:00:06.474220 12819 leader_election.cc:302] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93e6c924eebd4ca99eec02ff46236160, c6fd0b18618e4ecda64db5c7acb15b5d; no voters:
I20260809 01:00:06.474629 13123 raft_consensus.cc:2781] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:00:06.475106 13123 raft_consensus.cc:686] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 LEADER]: Becoming Leader. State: Replica: c6fd0b18618e4ecda64db5c7acb15b5d, State: Running, Role: LEADER
I20260809 01:00:06.475594 13123 consensus_queue.cc:237] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } }
I20260809 01:00:06.479986 12571 catalog_manager.cc:5495] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d reported cstate change: term changed from 0 to 1, leader changed from <none> to c6fd0b18618e4ecda64db5c7acb15b5d (127.4.82.130). New cstate: current_term: 1 leader_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } health_report { overall_health: UNKNOWN } } }
I20260809 01:00:06.547374 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.541886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55964 (local address 127.4.82.130:42037)
0809 01:00:06.542090 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.542096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.542116 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:00:06.542334 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.542338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.542433 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.542589 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:00:06.542956 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.543893 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.545211 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.545592 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.545754 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.546195 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:00:06.546734 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:00:06.546897 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.546980 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.547057 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 01:00:06.555441 13040 raft_consensus.cc:1261] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Refusing update from remote peer c6fd0b18618e4ecda64db5c7acb15b5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:06.555475 12741 raft_consensus.cc:1261] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Refusing update from remote peer c6fd0b18618e4ecda64db5c7acb15b5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:06.556497 13128 consensus_queue.cc:1035] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:00:06.556846 13123 consensus_queue.cc:1035] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:00:06.568819 13131 mvcc.cc:205] Tried to move back new op lower bound from 7316427598041354240 to 7316427597728043008. Current Snapshot: MvccSnapshot[applied={T|T < 7316427598041354240}]
I20260809 01:00:06.582679 12573 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:00:06.582945 12573 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:00:06.586336 12573 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4: Sending DeleteTablet for 3 replicas of tablet 298e607ac23b49e39afe51324f16f5ec
I20260809 01:00:06.587530 12868 tablet_service.cc:1514] Processing DeleteTablet for tablet 298e607ac23b49e39afe51324f16f5ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:06 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:55952
I20260809 01:00:06.587569 13015 tablet_service.cc:1514] Processing DeleteTablet for tablet 298e607ac23b49e39afe51324f16f5ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:06 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:52148
I20260809 01:00:06.587545 12720 tablet_service.cc:1514] Processing DeleteTablet for tablet 298e607ac23b49e39afe51324f16f5ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:06 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:42224
I20260809 01:00:06.589555 12573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:00:06.590380 12573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:00:06.601311 12718 tablet_service.cc:1467] Processing CreateTablet for tablet f365270924154f0bbee1aaf8796526da (DEFAULT_TABLE table=test-table [id=95d1bd43072c43d69f4e925baabf2acd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:06.601900 12718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f365270924154f0bbee1aaf8796526da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:06.601969 13014 tablet_service.cc:1467] Processing CreateTablet for tablet f365270924154f0bbee1aaf8796526da (DEFAULT_TABLE table=test-table [id=95d1bd43072c43d69f4e925baabf2acd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:06.602573 13014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f365270924154f0bbee1aaf8796526da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:06.607685 13142 tablet_replica.cc:330] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160: stopping tablet replica
I20260809 01:00:06.607841 12867 tablet_service.cc:1467] Processing CreateTablet for tablet f365270924154f0bbee1aaf8796526da (DEFAULT_TABLE table=test-table [id=95d1bd43072c43d69f4e925baabf2acd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:06.608143 13142 raft_consensus.cc:2229] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:00:06.608433 12867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f365270924154f0bbee1aaf8796526da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:06.608672 13142 raft_consensus.cc:2258] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:06.610819 13142 ts_tablet_manager.cc:1905] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:06.611207 13121 tablet_bootstrap.cc:492] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160: Bootstrap starting.
I20260809 01:00:06.612458 13141 tablet_replica.cc:330] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c: stopping tablet replica
I20260809 01:00:06.612849 13141 raft_consensus.cc:2229] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:00:06.613009 13121 tablet_bootstrap.cc:654] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:06.613363 13141 raft_consensus.cc:2258] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:06.614529 13143 tablet_replica.cc:330] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d: stopping tablet replica
I20260809 01:00:06.613440 13118 tablet_bootstrap.cc:492] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c: Bootstrap starting.
I20260809 01:00:06.616611 13121 tablet_bootstrap.cc:492] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160: No bootstrap required, opened a new log
I20260809 01:00:06.617033 13121 ts_tablet_manager.cc:1397] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 01:00:06.618131 13118 tablet_bootstrap.cc:654] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:06.620359 13121 raft_consensus.cc:348] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.620595 13121 raft_consensus.cc:374] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:06.620859 13121 raft_consensus.cc:729] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e6c924eebd4ca99eec02ff46236160, State: Initialized, Role: FOLLOWER
I20260809 01:00:06.620932 13142 ts_tablet_manager.cc:1918] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:00:06.621297 13142 log.cc:1196] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/298e607ac23b49e39afe51324f16f5ec
I20260809 01:00:06.621302 13121 consensus_queue.cc:260] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.622035 13142 ts_tablet_manager.cc:1939] T 298e607ac23b49e39afe51324f16f5ec P 93e6c924eebd4ca99eec02ff46236160: Deleting consensus metadata
I20260809 01:00:06.622207 13141 ts_tablet_manager.cc:1905] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:06.624356 12559 catalog_manager.cc:4841] TS 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131:34163): tablet 298e607ac23b49e39afe51324f16f5ec (table test-table [id=e1822129436d4c3b888fecff26352b1a]) successfully deleted
I20260809 01:00:06.625217 13121 ts_tablet_manager.cc:1428] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260809 01:00:06.626575 13118 tablet_bootstrap.cc:492] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c: No bootstrap required, opened a new log
I20260809 01:00:06.626796 13118 ts_tablet_manager.cc:1397] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.001s
I20260809 01:00:06.619037 13117 tablet_bootstrap.cc:492] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d: Bootstrap starting.
I20260809 01:00:06.627781 13143 raft_consensus.cc:2229] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:00:06.628479 13143 raft_consensus.cc:2258] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:06.628985 13141 ts_tablet_manager.cc:1918] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:00:06.629040 13117 tablet_bootstrap.cc:654] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:06.629128 13141 log.cc:1196] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/298e607ac23b49e39afe51324f16f5ec
I20260809 01:00:06.629658 13141 ts_tablet_manager.cc:1939] T 298e607ac23b49e39afe51324f16f5ec P 021fcab1a6da4afbbc78776ce563549c: Deleting consensus metadata
I20260809 01:00:06.630502 13143 ts_tablet_manager.cc:1905] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:06.630600 13118 raft_consensus.cc:348] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.630817 13118 raft_consensus.cc:374] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:06.630913 13118 raft_consensus.cc:729] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021fcab1a6da4afbbc78776ce563549c, State: Initialized, Role: FOLLOWER
I20260809 01:00:06.631356 12556 catalog_manager.cc:4841] TS 021fcab1a6da4afbbc78776ce563549c (127.4.82.129:43233): tablet 298e607ac23b49e39afe51324f16f5ec (table test-table [id=e1822129436d4c3b888fecff26352b1a]) successfully deleted
I20260809 01:00:06.631264 13118 consensus_queue.cc:260] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.635473 13118 ts_tablet_manager.cc:1428] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260809 01:00:06.638226 13143 ts_tablet_manager.cc:1918] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:00:06.638399 13143 log.cc:1196] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/298e607ac23b49e39afe51324f16f5ec
I20260809 01:00:06.638993 13143 ts_tablet_manager.cc:1939] T 298e607ac23b49e39afe51324f16f5ec P c6fd0b18618e4ecda64db5c7acb15b5d: Deleting consensus metadata
I20260809 01:00:06.640829 12559 catalog_manager.cc:4841] TS c6fd0b18618e4ecda64db5c7acb15b5d (127.4.82.130:42037): tablet 298e607ac23b49e39afe51324f16f5ec (table test-table [id=e1822129436d4c3b888fecff26352b1a]) successfully deleted
I20260809 01:00:06.640882 13117 tablet_bootstrap.cc:492] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d: No bootstrap required, opened a new log
I20260809 01:00:06.641040 13117 ts_tablet_manager.cc:1397] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d: Time spent bootstrapping tablet: real 0.022s user 0.011s sys 0.000s
I20260809 01:00:06.641660 13117 raft_consensus.cc:348] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.641877 13117 raft_consensus.cc:374] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:06.641987 13117 raft_consensus.cc:729] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6fd0b18618e4ecda64db5c7acb15b5d, State: Initialized, Role: FOLLOWER
I20260809 01:00:06.642251 13117 consensus_queue.cc:260] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.642740 13117 ts_tablet_manager.cc:1428] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 01:00:06.652611 13124 raft_consensus.cc:482] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:00:06.652817 13124 raft_consensus.cc:504] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.654559 13124 leader_election.cc:288] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6fd0b18618e4ecda64db5c7acb15b5d (127.4.82.130:42037), 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131:34163)
I20260809 01:00:06.662086 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.654859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:53081 (local address 127.4.82.130:42037)
0809 01:00:06.655034 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.655040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.655385 (+ 345us) server_negotiation.cc:407] Connection header received
0809 01:00:06.655676 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.655685 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.655794 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.655873 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:06.656612 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.657808 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.659880 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.661028 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.661185 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.661279 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.661619 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.661756 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 01:00:06.662892 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.655535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:34237 (local address 127.4.82.131:34163)
0809 01:00:06.655706 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.655711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.656305 (+ 594us) server_negotiation.cc:407] Connection header received
0809 01:00:06.656485 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.656491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.656589 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.656662 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:06.657171 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.657900 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.660494 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.661743 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.661889 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.661968 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.662478 (+ 510us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.662607 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 01:00:06.662971 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.654710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:42037 (local address 127.4.82.129:53081)
0809 01:00:06.655255 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:06.655277 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:06.655489 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:06.656070 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:06.656072 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:06.656092 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:06.656456 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.656468 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.657952 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.657960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.659710 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.659721 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.661245 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.661252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.661420 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.661454 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 01:00:06.662387 (+ 933us) client_negotiation.cc:240] Negotiation successful
0809 01:00:06.662531 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":77,"thread_start_us":284,"threads_started":1}
I20260809 01:00:06.663043 13148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.655434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:34163 (local address 127.4.82.129:34237)
0809 01:00:06.656171 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:06.656191 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:06.656339 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:06.656690 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:06.656702 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:06.656725 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:06.657022 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.657030 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.658015 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.658020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.660340 (+ 2320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.660357 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.662115 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.662121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.662306 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.662338 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 01:00:06.662451 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 01:00:06.662578 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":22,"thread_start_us":403,"threads_started":1}
I20260809 01:00:06.663892 12890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f365270924154f0bbee1aaf8796526da" candidate_uuid: "021fcab1a6da4afbbc78776ce563549c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" is_pre_election: true
I20260809 01:00:06.664286 12890 raft_consensus.cc:2454] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 021fcab1a6da4afbbc78776ce563549c in term 0.
I20260809 01:00:06.665248 13040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f365270924154f0bbee1aaf8796526da" candidate_uuid: "021fcab1a6da4afbbc78776ce563549c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93e6c924eebd4ca99eec02ff46236160" is_pre_election: true
I20260809 01:00:06.665220 12669 leader_election.cc:302] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 021fcab1a6da4afbbc78776ce563549c, c6fd0b18618e4ecda64db5c7acb15b5d; no voters:
I20260809 01:00:06.665504 13040 raft_consensus.cc:2454] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 021fcab1a6da4afbbc78776ce563549c in term 0.
I20260809 01:00:06.665616 13124 raft_consensus.cc:2781] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:00:06.665736 13124 raft_consensus.cc:482] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:00:06.665841 13124 raft_consensus.cc:3037] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:06.668116 13124 raft_consensus.cc:504] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.668815 13124 leader_election.cc:288] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [CANDIDATE]: Term 1 election: Requested vote from peers c6fd0b18618e4ecda64db5c7acb15b5d (127.4.82.130:42037), 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131:34163)
I20260809 01:00:06.669132 12890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f365270924154f0bbee1aaf8796526da" candidate_uuid: "021fcab1a6da4afbbc78776ce563549c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d"
I20260809 01:00:06.669353 12890 raft_consensus.cc:3037] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:06.669263 13040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f365270924154f0bbee1aaf8796526da" candidate_uuid: "021fcab1a6da4afbbc78776ce563549c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93e6c924eebd4ca99eec02ff46236160"
I20260809 01:00:06.669505 13040 raft_consensus.cc:3037] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:06.671579 13040 raft_consensus.cc:2454] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 021fcab1a6da4afbbc78776ce563549c in term 1.
I20260809 01:00:06.672081 12669 leader_election.cc:302] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 021fcab1a6da4afbbc78776ce563549c, 93e6c924eebd4ca99eec02ff46236160; no voters:
I20260809 01:00:06.672425 13124 raft_consensus.cc:2781] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:00:06.672590 12890 raft_consensus.cc:2454] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 021fcab1a6da4afbbc78776ce563549c in term 1.
I20260809 01:00:06.673264 13124 raft_consensus.cc:686] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [term 1 LEADER]: Becoming Leader. State: Replica: 021fcab1a6da4afbbc78776ce563549c, State: Running, Role: LEADER
I20260809 01:00:06.673671 13124 consensus_queue.cc:237] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.677141 12571 catalog_manager.cc:5495] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c reported cstate change: term changed from 0 to 1, leader changed from <none> to 021fcab1a6da4afbbc78776ce563549c (127.4.82.129). New cstate: current_term: 1 leader_uuid: "021fcab1a6da4afbbc78776ce563549c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } health_report { overall_health: UNKNOWN } } }
Aug 09 01:00:06 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237204, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.190@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 01:00:06.721256 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.705521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39594 (local address 127.4.82.190:43655)
0809 01:00:06.706546 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.706550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.706569 (+ 19us) server_negotiation.cc:407] Connection header received
0809 01:00:06.706611 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.706614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.706670 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.706756 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:06.708334 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.709154 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.711033 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.711517 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.711610 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.715492 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:06.715498 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:06.715501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:06.715528 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:06.718553 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:06.719221 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:06.719228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:06.719234 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:06.719416 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:06.719785 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:06.719792 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:06.719795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:06.720060 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:06.720215 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.720619 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.720823 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":944,"thread_start_us":763,"threads_started":1}
I20260809 01:00:06.721333 13157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.705489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:43655 (local address 127.0.0.1:39594)
0809 01:00:06.706189 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:06.706209 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:06.706358 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:06.706830 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:06.706836 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:06.707830 (+ 994us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:06.708150 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.708163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.709314 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.709320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.710882 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.710896 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.711684 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.711691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.711873 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.712624 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:06.712655 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:06.715239 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:06.718693 (+ 3454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:06.718705 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:06.718712 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:06.719085 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:06.719523 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:06.719529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:06.719532 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:06.719679 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:06.720184 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:06.720192 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:06.720354 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 01:00:06.720547 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 01:00:06.720947 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":145,"thread_start_us":383,"threads_started":1}
I20260809 01:00:06.726454 12571 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:39594:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 01:00:06.727509 12571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:00:06.734076 12720 tablet_service.cc:1467] Processing CreateTablet for tablet e832ba8dc0ee4acfbe359294e74a452d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bccc7ddf94a444aeb9adc684e94ade06]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 01:00:06.734457 12867 tablet_service.cc:1467] Processing CreateTablet for tablet e832ba8dc0ee4acfbe359294e74a452d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bccc7ddf94a444aeb9adc684e94ade06]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 01:00:06.734450 13014 tablet_service.cc:1467] Processing CreateTablet for tablet e832ba8dc0ee4acfbe359294e74a452d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bccc7ddf94a444aeb9adc684e94ade06]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 01:00:06.734692 12720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e832ba8dc0ee4acfbe359294e74a452d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:06.734961 13014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e832ba8dc0ee4acfbe359294e74a452d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:06.734973 12867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e832ba8dc0ee4acfbe359294e74a452d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:06.742048 13118 tablet_bootstrap.cc:492] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c: Bootstrap starting.
I20260809 01:00:06.742079 13117 tablet_bootstrap.cc:492] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d: Bootstrap starting.
I20260809 01:00:06.742370 13121 tablet_bootstrap.cc:492] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160: Bootstrap starting.
I20260809 01:00:06.743744 13118 tablet_bootstrap.cc:654] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:06.743848 13117 tablet_bootstrap.cc:654] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:06.744062 13121 tablet_bootstrap.cc:654] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:06.754343 13118 tablet_bootstrap.cc:492] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c: No bootstrap required, opened a new log
I20260809 01:00:06.754557 13118 ts_tablet_manager.cc:1397] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260809 01:00:06.755065 13117 tablet_bootstrap.cc:492] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d: No bootstrap required, opened a new log
I20260809 01:00:06.755334 13117 ts_tablet_manager.cc:1397] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260809 01:00:06.755321 13118 raft_consensus.cc:348] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.755640 13118 raft_consensus.cc:374] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:06.755764 13118 raft_consensus.cc:729] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021fcab1a6da4afbbc78776ce563549c, State: Initialized, Role: FOLLOWER
I20260809 01:00:06.756071 13118 consensus_queue.cc:260] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.756306 13117 raft_consensus.cc:348] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.756644 13117 raft_consensus.cc:374] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:06.756549 13124 tablet_replica.cc:439] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } }
I20260809 01:00:06.756742 13117 raft_consensus.cc:729] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6fd0b18618e4ecda64db5c7acb15b5d, State: Initialized, Role: FOLLOWER
I20260809 01:00:06.756815 13124 tablet_replica.cc:442] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 01:00:06.757040 13117 consensus_queue.cc:260] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.757769 13117 ts_tablet_manager.cc:1428] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 01:00:06.757560 13136 tablet_replica.cc:439] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } }
I20260809 01:00:06.757999 13136 tablet_replica.cc:442] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 01:00:06.760702 13121 tablet_bootstrap.cc:492] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160: No bootstrap required, opened a new log
I20260809 01:00:06.760746 13118 ts_tablet_manager.cc:1428] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260809 01:00:06.760890 13121 ts_tablet_manager.cc:1397] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.005s
I20260809 01:00:06.761732 13121 raft_consensus.cc:348] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.761972 13121 raft_consensus.cc:374] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:06.762056 13121 raft_consensus.cc:729] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e6c924eebd4ca99eec02ff46236160, State: Initialized, Role: FOLLOWER
I20260809 01:00:06.762300 13121 consensus_queue.cc:260] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.762868 13121 ts_tablet_manager.cc:1428] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 01:00:06.762759 13125 tablet_replica.cc:439] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } }
I20260809 01:00:06.763207 13125 tablet_replica.cc:442] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 01:00:06.835127 13136 raft_consensus.cc:482] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:00:06.835398 13136 raft_consensus.cc:504] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.836161 13136 leader_election.cc:288] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 021fcab1a6da4afbbc78776ce563549c (127.4.82.129:43233), 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131:34163)
I20260809 01:00:06.836735 13040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e832ba8dc0ee4acfbe359294e74a452d" candidate_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93e6c924eebd4ca99eec02ff46236160" is_pre_election: true
I20260809 01:00:06.836759 12741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e832ba8dc0ee4acfbe359294e74a452d" candidate_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "021fcab1a6da4afbbc78776ce563549c" is_pre_election: true
I20260809 01:00:06.837041 12741 raft_consensus.cc:2454] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6fd0b18618e4ecda64db5c7acb15b5d in term 0.
I20260809 01:00:06.837054 13040 raft_consensus.cc:2454] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6fd0b18618e4ecda64db5c7acb15b5d in term 0.
I20260809 01:00:06.837532 12816 leader_election.cc:302] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 021fcab1a6da4afbbc78776ce563549c, c6fd0b18618e4ecda64db5c7acb15b5d; no voters:
I20260809 01:00:06.837860 13136 raft_consensus.cc:2781] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:00:06.838016 13136 raft_consensus.cc:482] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:00:06.838137 13136 raft_consensus.cc:3037] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:06.840565 13136 raft_consensus.cc:504] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.841109 13136 leader_election.cc:288] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [CANDIDATE]: Term 1 election: Requested vote from peers 021fcab1a6da4afbbc78776ce563549c (127.4.82.129:43233), 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131:34163)
I20260809 01:00:06.841571 12741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e832ba8dc0ee4acfbe359294e74a452d" candidate_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "021fcab1a6da4afbbc78776ce563549c"
I20260809 01:00:06.841615 13040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e832ba8dc0ee4acfbe359294e74a452d" candidate_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93e6c924eebd4ca99eec02ff46236160"
I20260809 01:00:06.841770 12741 raft_consensus.cc:3037] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:06.841828 13040 raft_consensus.cc:3037] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:06.843981 12741 raft_consensus.cc:2454] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6fd0b18618e4ecda64db5c7acb15b5d in term 1.
I20260809 01:00:06.844401 12816 leader_election.cc:302] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 021fcab1a6da4afbbc78776ce563549c, c6fd0b18618e4ecda64db5c7acb15b5d; no voters:
I20260809 01:00:06.844681 13136 raft_consensus.cc:2781] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:00:06.844813 13040 raft_consensus.cc:2454] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6fd0b18618e4ecda64db5c7acb15b5d in term 1.
I20260809 01:00:06.844853 13136 raft_consensus.cc:686] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 LEADER]: Becoming Leader. State: Replica: c6fd0b18618e4ecda64db5c7acb15b5d, State: Running, Role: LEADER
I20260809 01:00:06.845149 13136 consensus_queue.cc:237] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:06.847010 13123 tablet_replica.cc:439] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d: TxnStatusTablet state changed. Reason: New leader c6fd0b18618e4ecda64db5c7acb15b5d. Latest consensus state: current_term: 1 leader_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } }
I20260809 01:00:06.847327 13123 tablet_replica.cc:442] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d: This TxnStatusTablet replica's current role is: LEADER
I20260809 01:00:06.848827 13162 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 01:00:06.849078 13162 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 01:00:06.849159 12571 catalog_manager.cc:5495] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d reported cstate change: term changed from 0 to 1, leader changed from <none> to c6fd0b18618e4ecda64db5c7acb15b5d (127.4.82.130). New cstate: current_term: 1 leader_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } health_report { overall_health: UNKNOWN } } }
I20260809 01:00:06.884904 13157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.879416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:42037 (local address 127.0.0.1:55968)
0809 01:00:06.879698 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:06.879716 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:06.879826 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:06.880221 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:06.880224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:06.880233 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 01:00:06.880378 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.880385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.881795 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.881799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.882773 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.882784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.883834 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.883839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.884009 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.884060 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 01:00:06.884492 (+ 432us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 01:00:06.884507 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 01:00:06.884567 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 01:00:06.884679 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 01:00:06.884992 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.879562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.4.82.130:42037)
0809 01:00:06.879769 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.879775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.879797 (+ 22us) server_negotiation.cc:407] Connection header received
0809 01:00:06.879999 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.880005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.880089 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.880192 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:00:06.880511 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.881666 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.882915 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.883688 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.883830 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.884260 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:00:06.884390 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:00:06.884484 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.884641 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.884723 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 01:00:06.888499 12741 raft_consensus.cc:1261] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Refusing update from remote peer c6fd0b18618e4ecda64db5c7acb15b5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:06.888574 13040 raft_consensus.cc:1261] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Refusing update from remote peer c6fd0b18618e4ecda64db5c7acb15b5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:06.889204 13136 consensus_queue.cc:1035] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:00:06.889487 13123 consensus_queue.cc:1035] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:00:06.892585 13124 tablet_replica.cc:439] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c: TxnStatusTablet state changed. Reason: New leader c6fd0b18618e4ecda64db5c7acb15b5d. Latest consensus state: current_term: 1 leader_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } }
I20260809 01:00:06.892907 13124 tablet_replica.cc:442] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 01:00:06.893898 13125 tablet_replica.cc:439] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160: TxnStatusTablet state changed. Reason: New leader c6fd0b18618e4ecda64db5c7acb15b5d. Latest consensus state: current_term: 1 leader_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } }
I20260809 01:00:06.894182 13125 tablet_replica.cc:442] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 01:00:06.895169 13136 tablet_replica.cc:439] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } }
I20260809 01:00:06.895478 13136 tablet_replica.cc:442] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d: This TxnStatusTablet replica's current role is: LEADER
I20260809 01:00:06.895325 13128 tablet_replica.cc:439] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } }
I20260809 01:00:06.895668 13128 tablet_replica.cc:442] T e832ba8dc0ee4acfbe359294e74a452d P c6fd0b18618e4ecda64db5c7acb15b5d: This TxnStatusTablet replica's current role is: LEADER
I20260809 01:00:06.898460 13125 tablet_replica.cc:439] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } }
I20260809 01:00:06.898561 13133 mvcc.cc:205] Tried to move back new op lower bound from 7316427599408996352 to 7316427599240613888. Current Snapshot: MvccSnapshot[applied={T|T < 7316427599408996352 or (T in {7316427599408996352})}]
I20260809 01:00:06.898682 13125 tablet_replica.cc:442] T e832ba8dc0ee4acfbe359294e74a452d P 93e6c924eebd4ca99eec02ff46236160: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 01:00:06.898873 13124 tablet_replica.cc:439] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } } }
I20260809 01:00:06.899108 13124 tablet_replica.cc:442] T e832ba8dc0ee4acfbe359294e74a452d P 021fcab1a6da4afbbc78776ce563549c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 01:00:06.910969 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.905156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42236 (local address 127.4.82.129:43233)
0809 01:00:06.905339 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.905345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.905366 (+ 21us) server_negotiation.cc:407] Connection header received
0809 01:00:06.905520 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.905527 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.905640 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.905796 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:00:06.906092 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.907199 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.908532 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.909155 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.909301 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.909686 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:00:06.910375 (+ 689us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:00:06.910515 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.910568 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.910652 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 01:00:06.927449 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.921056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:41795 (local address 127.4.82.130:42037)
0809 01:00:06.921253 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.921259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.921846 (+ 587us) server_negotiation.cc:407] Connection header received
0809 01:00:06.922022 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.922026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.922087 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.922171 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:00:06.922770 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.923735 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.925695 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.926136 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.926233 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.926743 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:00:06.926856 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:00:06.926919 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.927098 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.927168 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 01:00:06.927629 13168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.920960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:42037 (local address 127.4.82.129:41795)
0809 01:00:06.921732 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:06.921753 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:06.921903 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:06.922204 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:06.922210 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:06.922233 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 01:00:06.922605 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.922620 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.923873 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.923879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.925555 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.925570 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.926335 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.926342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.926509 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.926576 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 01:00:06.926963 (+ 387us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 01:00:06.926977 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 01:00:06.927072 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 01:00:06.927237 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":451,"threads_started":1}
I20260809 01:00:06.937031 13039 raft_consensus.cc:1261] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Refusing update from remote peer 021fcab1a6da4afbbc78776ce563549c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:06.937297 12890 raft_consensus.cc:1261] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Refusing update from remote peer 021fcab1a6da4afbbc78776ce563549c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:06.937727 13149 consensus_queue.cc:1035] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [LEADER]: Connected to new peer: Peer: permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:00:06.938097 13124 consensus_queue.cc:1035] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:00:06.979656 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.972382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:39703 (local address 127.4.82.129:43233)
0809 01:00:06.972594 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:06.972600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:06.973290 (+ 690us) server_negotiation.cc:407] Connection header received
0809 01:00:06.973383 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:06.973389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:06.973477 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:06.973578 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:00:06.974241 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.975217 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.977854 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.978252 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.978345 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.978892 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:00:06.979021 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:00:06.979133 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:06.979257 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 01:00:06.979357 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 01:00:06.979816 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:06.972301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:43233 (local address 127.4.82.130:39703)
0809 01:00:06.972992 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:06.973015 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:06.973159 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:06.973607 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:06.973612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:06.973624 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 01:00:06.974005 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.974021 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.975813 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.975818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.977704 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:06.977721 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:06.978444 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:06.978451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:06.978636 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:06.978704 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 01:00:06.979138 (+ 434us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 01:00:06.979153 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 01:00:06.979225 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 01:00:06.979394 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":342,"threads_started":1}
I20260809 01:00:07.033659 12570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:00:07.033942 12570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:00:07.037768 12570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4: Sending DeleteTablet for 3 replicas of tablet f365270924154f0bbee1aaf8796526da
I20260809 01:00:07.038561 12720 tablet_service.cc:1514] Processing DeleteTablet for tablet f365270924154f0bbee1aaf8796526da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:07 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:42224
I20260809 01:00:07.038861 13141 tablet_replica.cc:330] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c: stopping tablet replica
I20260809 01:00:07.038969 13014 tablet_service.cc:1514] Processing DeleteTablet for tablet f365270924154f0bbee1aaf8796526da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:07 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:52148
I20260809 01:00:07.039109 13141 raft_consensus.cc:2229] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:00:07.039325 13142 tablet_replica.cc:330] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160: stopping tablet replica
I20260809 01:00:07.039533 13142 raft_consensus.cc:2229] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:00:07.039649 13141 raft_consensus.cc:2258] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:07.039762 13142 raft_consensus.cc:2258] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:07.041226 13141 ts_tablet_manager.cc:1905] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:07.041358 13142 ts_tablet_manager.cc:1905] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:07.043658 12867 tablet_service.cc:1514] Processing DeleteTablet for tablet f365270924154f0bbee1aaf8796526da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:07 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:55952
I20260809 01:00:07.047477 13143 tablet_replica.cc:330] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d: stopping tablet replica
I20260809 01:00:07.047712 13143 raft_consensus.cc:2229] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:00:07.047945 13143 raft_consensus.cc:2258] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:07.049196 13143 ts_tablet_manager.cc:1905] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:07.057857 13141 ts_tablet_manager.cc:1918] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 01:00:07.057994 13143 ts_tablet_manager.cc:1918] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 01:00:07.058229 13143 log.cc:1196] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/f365270924154f0bbee1aaf8796526da
I20260809 01:00:07.058553 13142 ts_tablet_manager.cc:1918] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 01:00:07.058720 13142 log.cc:1196] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/f365270924154f0bbee1aaf8796526da
I20260809 01:00:07.059053 13143 ts_tablet_manager.cc:1939] T f365270924154f0bbee1aaf8796526da P c6fd0b18618e4ecda64db5c7acb15b5d: Deleting consensus metadata
I20260809 01:00:07.059166 13142 ts_tablet_manager.cc:1939] T f365270924154f0bbee1aaf8796526da P 93e6c924eebd4ca99eec02ff46236160: Deleting consensus metadata
I20260809 01:00:07.059401 13141 log.cc:1196] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/f365270924154f0bbee1aaf8796526da
I20260809 01:00:07.060735 13141 ts_tablet_manager.cc:1939] T f365270924154f0bbee1aaf8796526da P 021fcab1a6da4afbbc78776ce563549c: Deleting consensus metadata
I20260809 01:00:07.061892 12556 catalog_manager.cc:4841] TS 021fcab1a6da4afbbc78776ce563549c (127.4.82.129:43233): tablet f365270924154f0bbee1aaf8796526da (table test-table [id=95d1bd43072c43d69f4e925baabf2acd]) successfully deleted
I20260809 01:00:07.062391 12559 catalog_manager.cc:4841] TS c6fd0b18618e4ecda64db5c7acb15b5d (127.4.82.130:42037): tablet f365270924154f0bbee1aaf8796526da (table test-table [id=95d1bd43072c43d69f4e925baabf2acd]) successfully deleted
I20260809 01:00:07.065718 12559 catalog_manager.cc:4841] TS 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131:34163): tablet f365270924154f0bbee1aaf8796526da (table test-table [id=95d1bd43072c43d69f4e925baabf2acd]) successfully deleted
Aug 09 01:00:07 dist-test-slave-07c2 krb5kdc[12526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 01:00:07 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:07 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 01:00:07.138090 13156 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:07.120426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39596 (local address 127.4.82.190:43655)
0809 01:00:07.120611 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:07.120618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:07.121124 (+ 506us) server_negotiation.cc:407] Connection header received
0809 01:00:07.121392 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:07.121401 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:07.121526 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:07.121617 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:07.123489 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.124650 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.126511 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.127202 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.127371 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:07.137836 (+ 10465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39596: BlockingRecv error: recv got EOF from 127.0.0.1:39596 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 01:00:07 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM
I20260809 01:00:07.159049 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:07.141737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39600 (local address 127.4.82.190:43655)
0809 01:00:07.141911 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:07.141916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:07.142373 (+ 457us) server_negotiation.cc:407] Connection header received
0809 01:00:07.142534 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:07.142539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:07.142617 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:07.142717 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:07.144334 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.145410 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.147195 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.147907 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.148047 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:07.152593 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:07.152601 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:07.152606 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:07.152643 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:07.156162 (+ 3519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:07.156914 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:07.156919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:07.156923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:07.157092 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:07.157545 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:07.157550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:07.157553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:07.157880 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:07.158138 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:07.158433 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 01:00:07.158643 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 01:00:07.163017 12570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:00:07.163808 12570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:00:07.171226 12720 tablet_service.cc:1467] Processing CreateTablet for tablet dc68ddcfb92e4a78b86fd002faea6f9b (DEFAULT_TABLE table=test-table [id=43df8ed3c01d4b77a621a90009058634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:07.171744 12720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc68ddcfb92e4a78b86fd002faea6f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:07.171928 13014 tablet_service.cc:1467] Processing CreateTablet for tablet dc68ddcfb92e4a78b86fd002faea6f9b (DEFAULT_TABLE table=test-table [id=43df8ed3c01d4b77a621a90009058634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:07.172375 13014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc68ddcfb92e4a78b86fd002faea6f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:07.178678 13118 tablet_bootstrap.cc:492] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c: Bootstrap starting.
I20260809 01:00:07.178987 12867 tablet_service.cc:1467] Processing CreateTablet for tablet dc68ddcfb92e4a78b86fd002faea6f9b (DEFAULT_TABLE table=test-table [id=43df8ed3c01d4b77a621a90009058634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:07.179493 12867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc68ddcfb92e4a78b86fd002faea6f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:07.180414 13118 tablet_bootstrap.cc:654] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:07.186635 13121 tablet_bootstrap.cc:492] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160: Bootstrap starting.
I20260809 01:00:07.188308 13121 tablet_bootstrap.cc:654] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:07.190920 13118 tablet_bootstrap.cc:492] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c: No bootstrap required, opened a new log
I20260809 01:00:07.191130 13118 ts_tablet_manager.cc:1397] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260809 01:00:07.191811 13117 tablet_bootstrap.cc:492] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d: Bootstrap starting.
I20260809 01:00:07.191956 13118 raft_consensus.cc:348] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.192225 13118 raft_consensus.cc:374] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:07.192319 13118 raft_consensus.cc:729] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021fcab1a6da4afbbc78776ce563549c, State: Initialized, Role: FOLLOWER
I20260809 01:00:07.192566 13118 consensus_queue.cc:260] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.193470 13117 tablet_bootstrap.cc:654] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:07.195546 13118 ts_tablet_manager.cc:1428] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 01:00:07.201591 13121 tablet_bootstrap.cc:492] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160: No bootstrap required, opened a new log
I20260809 01:00:07.201789 13121 ts_tablet_manager.cc:1397] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.004s
I20260809 01:00:07.202487 13117 tablet_bootstrap.cc:492] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d: No bootstrap required, opened a new log
I20260809 01:00:07.203058 13121 raft_consensus.cc:348] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.203486 13121 raft_consensus.cc:374] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:07.203577 13117 ts_tablet_manager.cc:1397] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.006s
I20260809 01:00:07.203696 13121 raft_consensus.cc:729] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e6c924eebd4ca99eec02ff46236160, State: Initialized, Role: FOLLOWER
I20260809 01:00:07.203958 13121 consensus_queue.cc:260] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.204509 13117 raft_consensus.cc:348] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.205201 13121 ts_tablet_manager.cc:1428] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 01:00:07.204826 13117 raft_consensus.cc:374] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:07.206406 13117 raft_consensus.cc:729] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6fd0b18618e4ecda64db5c7acb15b5d, State: Initialized, Role: FOLLOWER
I20260809 01:00:07.206831 13117 consensus_queue.cc:260] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.210373 13117 ts_tablet_manager.cc:1428] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260809 01:00:07.500417 13125 raft_consensus.cc:482] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:00:07.500677 13125 raft_consensus.cc:504] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.502136 13125 leader_election.cc:288] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 021fcab1a6da4afbbc78776ce563549c (127.4.82.129:43233), c6fd0b18618e4ecda64db5c7acb15b5d (127.4.82.130:42037)
I20260809 01:00:07.525015 13192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:07.502300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:43233 (local address 127.4.82.131:54615)
0809 01:00:07.502818 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:07.502839 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:07.502975 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:07.518415 (+ 15440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:07.518421 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:07.518451 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:07.518691 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:07.518704 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.520218 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.520223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:07.522546 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:07.522562 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.524265 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.524272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:07.524461 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:07.524498 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 01:00:07.524560 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 01:00:07.524691 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":568,"threads_started":1}
I20260809 01:00:07.526182 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:07.502747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:54615 (local address 127.4.82.129:43233)
0809 01:00:07.518005 (+ 15258us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:07.518011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:07.518029 (+ 18us) server_negotiation.cc:407] Connection header received
0809 01:00:07.518090 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:07.518094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:07.518179 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:07.518829 (+ 650us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:07.518976 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.520042 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.522769 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.524052 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.525667 (+ 1615us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:07.525738 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:07.525781 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 01:00:07.525913 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15144,"thread_start_us":15511,"threads_started":1}
I20260809 01:00:07.526695 12741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc68ddcfb92e4a78b86fd002faea6f9b" candidate_uuid: "93e6c924eebd4ca99eec02ff46236160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "021fcab1a6da4afbbc78776ce563549c" is_pre_election: true
I20260809 01:00:07.526921 12741 raft_consensus.cc:2454] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93e6c924eebd4ca99eec02ff46236160 in term 0.
I20260809 01:00:07.527482 12962 leader_election.cc:302] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 021fcab1a6da4afbbc78776ce563549c, 93e6c924eebd4ca99eec02ff46236160; no voters:
I20260809 01:00:07.527837 13125 raft_consensus.cc:2781] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:00:07.527966 13125 raft_consensus.cc:482] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:00:07.528071 13125 raft_consensus.cc:3037] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:07.530807 13125 raft_consensus.cc:504] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.531842 12741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc68ddcfb92e4a78b86fd002faea6f9b" candidate_uuid: "93e6c924eebd4ca99eec02ff46236160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "021fcab1a6da4afbbc78776ce563549c"
I20260809 01:00:07.532047 12741 raft_consensus.cc:3037] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:07.534013 12741 raft_consensus.cc:2454] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93e6c924eebd4ca99eec02ff46236160 in term 1.
I20260809 01:00:07.531675 13125 leader_election.cc:288] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [CANDIDATE]: Term 1 election: Requested vote from peers 021fcab1a6da4afbbc78776ce563549c (127.4.82.129:43233), c6fd0b18618e4ecda64db5c7acb15b5d (127.4.82.130:42037)
I20260809 01:00:07.534447 12962 leader_election.cc:302] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 021fcab1a6da4afbbc78776ce563549c, 93e6c924eebd4ca99eec02ff46236160; no voters:
I20260809 01:00:07.534744 13125 raft_consensus.cc:2781] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:00:07.535142 13125 raft_consensus.cc:686] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [term 1 LEADER]: Becoming Leader. State: Replica: 93e6c924eebd4ca99eec02ff46236160, State: Running, Role: LEADER
I20260809 01:00:07.535926 13125 consensus_queue.cc:237] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.536981 13192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:07.502400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:42037 (local address 127.4.82.131:43339)
0809 01:00:07.525394 (+ 22994us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:07.525411 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:07.525499 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:07.527877 (+ 2378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:07.527882 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:07.527909 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:07.528073 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:07.528084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.529550 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.529556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:07.534492 (+ 4936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:07.534505 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.536200 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.536206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:07.536401 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:07.536434 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 01:00:07.536524 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 01:00:07.536657 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22889,"thread_start_us":27351,"threads_started":1}
I20260809 01:00:07.537091 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:07.502745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:43339 (local address 127.4.82.130:42037)
0809 01:00:07.527557 (+ 24812us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:07.527564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:07.527585 (+ 21us) server_negotiation.cc:407] Connection header received
0809 01:00:07.527643 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:07.527648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:07.527721 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:07.528201 (+ 480us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:07.528357 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.529380 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.534715 (+ 5335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.535998 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.536174 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:07.536256 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:07.536575 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 01:00:07.536730 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24693,"thread_start_us":24546,"threads_started":1}
I20260809 01:00:07.537914 12890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc68ddcfb92e4a78b86fd002faea6f9b" candidate_uuid: "93e6c924eebd4ca99eec02ff46236160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" is_pre_election: true
I20260809 01:00:07.538149 12890 raft_consensus.cc:2454] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93e6c924eebd4ca99eec02ff46236160 in term 0.
I20260809 01:00:07.538290 12891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc68ddcfb92e4a78b86fd002faea6f9b" candidate_uuid: "93e6c924eebd4ca99eec02ff46236160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d"
I20260809 01:00:07.538587 12891 raft_consensus.cc:3037] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:07.541329 12891 raft_consensus.cc:2454] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93e6c924eebd4ca99eec02ff46236160 in term 1.
I20260809 01:00:07.542626 12570 catalog_manager.cc:5495] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131). New cstate: current_term: 1 leader_uuid: "93e6c924eebd4ca99eec02ff46236160" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } health_report { overall_health: HEALTHY } } }
I20260809 01:00:07.583091 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:07.574742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52154 (local address 127.4.82.131:34163)
0809 01:00:07.576376 (+ 1634us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:07.576381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:07.576405 (+ 24us) server_negotiation.cc:407] Connection header received
0809 01:00:07.576529 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:07.576534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:07.576659 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:07.576833 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:00:07.577395 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.578519 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.580536 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.581309 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.581449 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:07.581937 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:00:07.582467 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:00:07.582632 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:07.582699 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 01:00:07.582774 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1509,"thread_start_us":1348,"threads_started":1}
I20260809 01:00:07.586972 12741 raft_consensus.cc:1261] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Refusing update from remote peer 93e6c924eebd4ca99eec02ff46236160: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:07.587081 12891 raft_consensus.cc:1261] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Refusing update from remote peer 93e6c924eebd4ca99eec02ff46236160: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:07.587761 13125 consensus_queue.cc:1035] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [LEADER]: Connected to new peer: Peer: permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:00:07.588078 13197 consensus_queue.cc:1035] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:00:07.593421 13201 mvcc.cc:205] Tried to move back new op lower bound from 7316427602269380608 to 7316427602072883200. Current Snapshot: MvccSnapshot[applied={T|T < 7316427602269380608}]
I20260809 01:00:07.605471 12571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:00:07.605733 12571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:00:07.609397 12571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4: Sending DeleteTablet for 3 replicas of tablet dc68ddcfb92e4a78b86fd002faea6f9b
I20260809 01:00:07.610384 13014 tablet_service.cc:1514] Processing DeleteTablet for tablet dc68ddcfb92e4a78b86fd002faea6f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:07 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:52148
I20260809 01:00:07.611014 12867 tablet_service.cc:1514] Processing DeleteTablet for tablet dc68ddcfb92e4a78b86fd002faea6f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:07 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:55952
I20260809 01:00:07.611687 12720 tablet_service.cc:1514] Processing DeleteTablet for tablet dc68ddcfb92e4a78b86fd002faea6f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:07 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:42224
I20260809 01:00:07.613559 12570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:00:07.614220 12570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:00:07.619683 13214 tablet_replica.cc:330] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c: stopping tablet replica
I20260809 01:00:07.619906 13214 raft_consensus.cc:2229] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:00:07.620242 13214 raft_consensus.cc:2258] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:07.621419 13214 ts_tablet_manager.cc:1905] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:07.621924 13212 tablet_replica.cc:330] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160: stopping tablet replica
I20260809 01:00:07.622689 12868 tablet_service.cc:1467] Processing CreateTablet for tablet d80f8d3ffc614120ab5a6288d1ac4be6 (DEFAULT_TABLE table=test-table [id=b961cf11dd6a49178ea5dae0e2d29da6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:07.622951 12720 tablet_service.cc:1467] Processing CreateTablet for tablet d80f8d3ffc614120ab5a6288d1ac4be6 (DEFAULT_TABLE table=test-table [id=b961cf11dd6a49178ea5dae0e2d29da6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:07.623168 12868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d80f8d3ffc614120ab5a6288d1ac4be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:07.623374 12720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d80f8d3ffc614120ab5a6288d1ac4be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:07.629824 13118 tablet_bootstrap.cc:492] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c: Bootstrap starting.
I20260809 01:00:07.629896 13117 tablet_bootstrap.cc:492] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d: Bootstrap starting.
I20260809 01:00:07.631570 13118 tablet_bootstrap.cc:654] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:07.631642 13117 tablet_bootstrap.cc:654] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:07.633559 13117 tablet_bootstrap.cc:492] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d: No bootstrap required, opened a new log
I20260809 01:00:07.633807 13117 ts_tablet_manager.cc:1397] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 01:00:07.634600 13117 raft_consensus.cc:348] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.635954 13117 raft_consensus.cc:374] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:07.636118 13117 raft_consensus.cc:729] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6fd0b18618e4ecda64db5c7acb15b5d, State: Initialized, Role: FOLLOWER
I20260809 01:00:07.636438 13117 consensus_queue.cc:260] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.623183 13014 tablet_service.cc:1467] Processing CreateTablet for tablet d80f8d3ffc614120ab5a6288d1ac4be6 (DEFAULT_TABLE table=test-table [id=b961cf11dd6a49178ea5dae0e2d29da6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:07.637820 13014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d80f8d3ffc614120ab5a6288d1ac4be6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:07.639490 13117 ts_tablet_manager.cc:1428] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260809 01:00:07.644425 13213 tablet_replica.cc:330] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d: stopping tablet replica
I20260809 01:00:07.645314 13213 raft_consensus.cc:2229] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:00:07.645596 13213 raft_consensus.cc:2258] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:07.645709 13118 tablet_bootstrap.cc:492] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c: No bootstrap required, opened a new log
I20260809 01:00:07.645921 13118 ts_tablet_manager.cc:1397] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.005s
I20260809 01:00:07.646379 13212 raft_consensus.cc:2229] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:00:07.646571 13213 ts_tablet_manager.cc:1905] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:07.646718 13118 raft_consensus.cc:348] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.647017 13118 raft_consensus.cc:374] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:07.647017 13212 raft_consensus.cc:2258] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:07.647179 13118 raft_consensus.cc:729] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021fcab1a6da4afbbc78776ce563549c, State: Initialized, Role: FOLLOWER
I20260809 01:00:07.647651 13121 tablet_bootstrap.cc:492] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160: Bootstrap starting.
I20260809 01:00:07.647507 13118 consensus_queue.cc:260] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.648177 13212 ts_tablet_manager.cc:1905] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:07.650209 13118 ts_tablet_manager.cc:1428] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 01:00:07.650569 13121 tablet_bootstrap.cc:654] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:07.654109 13213 ts_tablet_manager.cc:1918] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:00:07.654237 13213 log.cc:1196] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/dc68ddcfb92e4a78b86fd002faea6f9b
I20260809 01:00:07.654264 13121 tablet_bootstrap.cc:492] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160: No bootstrap required, opened a new log
I20260809 01:00:07.654431 13121 ts_tablet_manager.cc:1397] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260809 01:00:07.654644 13212 ts_tablet_manager.cc:1918] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:00:07.654760 13213 ts_tablet_manager.cc:1939] T dc68ddcfb92e4a78b86fd002faea6f9b P c6fd0b18618e4ecda64db5c7acb15b5d: Deleting consensus metadata
I20260809 01:00:07.654773 13212 log.cc:1196] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/dc68ddcfb92e4a78b86fd002faea6f9b
I20260809 01:00:07.655215 13212 ts_tablet_manager.cc:1939] T dc68ddcfb92e4a78b86fd002faea6f9b P 93e6c924eebd4ca99eec02ff46236160: Deleting consensus metadata
I20260809 01:00:07.655262 13121 raft_consensus.cc:348] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.655717 13121 raft_consensus.cc:374] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:07.655877 13121 raft_consensus.cc:729] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e6c924eebd4ca99eec02ff46236160, State: Initialized, Role: FOLLOWER
I20260809 01:00:07.655900 12559 catalog_manager.cc:4841] TS c6fd0b18618e4ecda64db5c7acb15b5d (127.4.82.130:42037): tablet dc68ddcfb92e4a78b86fd002faea6f9b (table test-table [id=43df8ed3c01d4b77a621a90009058634]) successfully deleted
I20260809 01:00:07.656291 12559 catalog_manager.cc:4841] TS 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131:34163): tablet dc68ddcfb92e4a78b86fd002faea6f9b (table test-table [id=43df8ed3c01d4b77a621a90009058634]) successfully deleted
I20260809 01:00:07.656167 13121 consensus_queue.cc:260] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.656797 13121 ts_tablet_manager.cc:1428] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 01:00:07.659507 13214 ts_tablet_manager.cc:1918] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:00:07.659713 13214 log.cc:1196] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/dc68ddcfb92e4a78b86fd002faea6f9b
I20260809 01:00:07.660261 13214 ts_tablet_manager.cc:1939] T dc68ddcfb92e4a78b86fd002faea6f9b P 021fcab1a6da4afbbc78776ce563549c: Deleting consensus metadata
I20260809 01:00:07.661284 12556 catalog_manager.cc:4841] TS 021fcab1a6da4afbbc78776ce563549c (127.4.82.129:43233): tablet dc68ddcfb92e4a78b86fd002faea6f9b (table test-table [id=43df8ed3c01d4b77a621a90009058634]) successfully deleted
I20260809 01:00:07.683846 13149 raft_consensus.cc:482] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:00:07.684056 13149 raft_consensus.cc:504] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.684721 13149 leader_election.cc:288] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6fd0b18618e4ecda64db5c7acb15b5d (127.4.82.130:42037), 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131:34163)
I20260809 01:00:07.685263 12891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80f8d3ffc614120ab5a6288d1ac4be6" candidate_uuid: "021fcab1a6da4afbbc78776ce563549c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" is_pre_election: true
I20260809 01:00:07.685360 13040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80f8d3ffc614120ab5a6288d1ac4be6" candidate_uuid: "021fcab1a6da4afbbc78776ce563549c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93e6c924eebd4ca99eec02ff46236160" is_pre_election: true
I20260809 01:00:07.685518 12891 raft_consensus.cc:2454] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 021fcab1a6da4afbbc78776ce563549c in term 0.
I20260809 01:00:07.685679 13040 raft_consensus.cc:2454] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 021fcab1a6da4afbbc78776ce563549c in term 0.
I20260809 01:00:07.686102 12669 leader_election.cc:302] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 021fcab1a6da4afbbc78776ce563549c, c6fd0b18618e4ecda64db5c7acb15b5d; no voters:
I20260809 01:00:07.686424 13149 raft_consensus.cc:2781] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:00:07.686544 13149 raft_consensus.cc:482] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:00:07.686648 13149 raft_consensus.cc:3037] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:07.689013 13149 raft_consensus.cc:504] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.689590 13149 leader_election.cc:288] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [CANDIDATE]: Term 1 election: Requested vote from peers c6fd0b18618e4ecda64db5c7acb15b5d (127.4.82.130:42037), 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131:34163)
I20260809 01:00:07.690151 12891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80f8d3ffc614120ab5a6288d1ac4be6" candidate_uuid: "021fcab1a6da4afbbc78776ce563549c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d"
I20260809 01:00:07.690171 13040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d80f8d3ffc614120ab5a6288d1ac4be6" candidate_uuid: "021fcab1a6da4afbbc78776ce563549c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93e6c924eebd4ca99eec02ff46236160"
I20260809 01:00:07.690356 12891 raft_consensus.cc:3037] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:07.690414 13040 raft_consensus.cc:3037] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:07.692730 13040 raft_consensus.cc:2454] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 021fcab1a6da4afbbc78776ce563549c in term 1.
I20260809 01:00:07.693249 12669 leader_election.cc:302] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 021fcab1a6da4afbbc78776ce563549c, 93e6c924eebd4ca99eec02ff46236160; no voters:
I20260809 01:00:07.693542 13149 raft_consensus.cc:2781] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:00:07.693989 13149 raft_consensus.cc:686] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [term 1 LEADER]: Becoming Leader. State: Replica: 021fcab1a6da4afbbc78776ce563549c, State: Running, Role: LEADER
I20260809 01:00:07.694324 13149 consensus_queue.cc:237] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } }
I20260809 01:00:07.694641 12891 raft_consensus.cc:2454] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 021fcab1a6da4afbbc78776ce563549c in term 1.
I20260809 01:00:07.700941 12570 catalog_manager.cc:5495] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c reported cstate change: term changed from 0 to 1, leader changed from <none> to 021fcab1a6da4afbbc78776ce563549c (127.4.82.129). New cstate: current_term: 1 leader_uuid: "021fcab1a6da4afbbc78776ce563549c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "021fcab1a6da4afbbc78776ce563549c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 43233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 } health_report { overall_health: UNKNOWN } } }
I20260809 01:00:07.742203 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:07.737056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42252 (local address 127.4.82.129:43233)
0809 01:00:07.737219 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:07.737225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:07.737688 (+ 463us) server_negotiation.cc:407] Connection header received
0809 01:00:07.737833 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:07.737840 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:07.737973 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:07.738063 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:00:07.738412 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.739223 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.740547 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.740951 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.741095 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:07.741500 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:00:07.741613 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:00:07.741708 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:07.741862 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 01:00:07.741934 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 01:00:07.751106 12891 raft_consensus.cc:1261] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Refusing update from remote peer 021fcab1a6da4afbbc78776ce563549c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:07.751153 13040 raft_consensus.cc:1261] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Refusing update from remote peer 021fcab1a6da4afbbc78776ce563549c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:07.751754 13218 consensus_queue.cc:1035] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [LEADER]: Connected to new peer: Peer: permanent_uuid: "93e6c924eebd4ca99eec02ff46236160" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:00:07.752121 13149 consensus_queue.cc:1035] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6fd0b18618e4ecda64db5c7acb15b5d" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:00:07.806106 12570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:00:07.806375 12570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:00:07.809386 12570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ded9735aa1764e249ba71df292682ec4: Sending DeleteTablet for 3 replicas of tablet d80f8d3ffc614120ab5a6288d1ac4be6
I20260809 01:00:07.810065 12720 tablet_service.cc:1514] Processing DeleteTablet for tablet d80f8d3ffc614120ab5a6288d1ac4be6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:07 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:42224
I20260809 01:00:07.810318 13214 tablet_replica.cc:330] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c: stopping tablet replica
I20260809 01:00:07.810282 13014 tablet_service.cc:1514] Processing DeleteTablet for tablet d80f8d3ffc614120ab5a6288d1ac4be6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:07 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:52148
I20260809 01:00:07.810441 12867 tablet_service.cc:1514] Processing DeleteTablet for tablet d80f8d3ffc614120ab5a6288d1ac4be6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:07 UTC) from {username='oryx', principal='oryx/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:55952
I20260809 01:00:07.810511 13214 raft_consensus.cc:2229] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:00:07.810563 13212 tablet_replica.cc:330] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160: stopping tablet replica
I20260809 01:00:07.810714 13213 tablet_replica.cc:330] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d: stopping tablet replica
I20260809 01:00:07.810743 13212 raft_consensus.cc:2229] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:00:07.810832 13214 raft_consensus.cc:2258] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:07.810848 13213 raft_consensus.cc:2229] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:00:07.810904 13212 raft_consensus.cc:2258] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:07.810966 13213 raft_consensus.cc:2258] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:07.812124 13213 ts_tablet_manager.cc:1905] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:07.812234 13212 ts_tablet_manager.cc:1905] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:07.813447 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 12649
I20260809 01:00:07.814805 13214 ts_tablet_manager.cc:1905] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:07.818641 13213 ts_tablet_manager.cc:1918] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 01:00:07.818889 13213 log.cc:1196] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/d80f8d3ffc614120ab5a6288d1ac4be6
I20260809 01:00:07.819391 13213 ts_tablet_manager.cc:1939] T d80f8d3ffc614120ab5a6288d1ac4be6 P c6fd0b18618e4ecda64db5c7acb15b5d: Deleting consensus metadata
I20260809 01:00:07.820472 12559 catalog_manager.cc:4841] TS c6fd0b18618e4ecda64db5c7acb15b5d (127.4.82.130:42037): tablet d80f8d3ffc614120ab5a6288d1ac4be6 (table test-table [id=b961cf11dd6a49178ea5dae0e2d29da6]) successfully deleted
I20260809 01:00:07.823616 13214 ts_tablet_manager.cc:1918] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 01:00:07.826385 13212 ts_tablet_manager.cc:1918] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 01:00:07.826575 13212 log.cc:1196] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/d80f8d3ffc614120ab5a6288d1ac4be6
I20260809 01:00:07.826998 13212 ts_tablet_manager.cc:1939] T d80f8d3ffc614120ab5a6288d1ac4be6 P 93e6c924eebd4ca99eec02ff46236160: Deleting consensus metadata
I20260809 01:00:07.828033 13214 log.cc:1196] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/d80f8d3ffc614120ab5a6288d1ac4be6
I20260809 01:00:07.828157 12559 catalog_manager.cc:4841] TS 93e6c924eebd4ca99eec02ff46236160 (127.4.82.131:34163): tablet d80f8d3ffc614120ab5a6288d1ac4be6 (table test-table [id=b961cf11dd6a49178ea5dae0e2d29da6]) successfully deleted
I20260809 01:00:07.828516 13214 ts_tablet_manager.cc:1939] T d80f8d3ffc614120ab5a6288d1ac4be6 P 021fcab1a6da4afbbc78776ce563549c: Deleting consensus metadata
I20260809 01:00:07.829389 12556 catalog_manager.cc:4841] TS 021fcab1a6da4afbbc78776ce563549c (127.4.82.129:43233): tablet d80f8d3ffc614120ab5a6288d1ac4be6 (table test-table [id=b961cf11dd6a49178ea5dae0e2d29da6]) successfully deleted
Aug 09 01:00:07 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.82.129@KRBTEST.COM
I20260809 01:00:07.840219 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:07.814807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42262 (local address 127.4.82.129:43233)
0809 01:00:07.817149 (+ 2342us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:07.817155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:07.823778 (+ 6623us) server_negotiation.cc:407] Connection header received
0809 01:00:07.824039 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:07.824046 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:07.824125 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:07.824245 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:07.825798 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.826913 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.828734 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.829398 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.829526 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:07.833788 (+ 4262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:07.833805 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:07.833825 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:07.833863 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:07.837499 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:07.838109 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:07.838117 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:07.838120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:07.838262 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:07.838620 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:07.838627 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:07.838630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:07.839056 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:07.839436 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:07.839659 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 01:00:07.839866 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2211}
I20260809 01:00:07.840891 12788 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:07.968226 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 12649
I20260809 01:00:07.993924 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 12797
Aug 09 01:00:07 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.82.130@KRBTEST.COM
I20260809 01:00:08.009634 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:07.994789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55980 (local address 127.4.82.130:42037)
0809 01:00:07.995023 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:07.995029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:07.995050 (+ 21us) server_negotiation.cc:407] Connection header received
0809 01:00:07.995227 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:07.995239 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:07.995364 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:07.995492 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:07.996805 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.997544 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.998877 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:07.999393 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:07.999533 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:08.003885 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:08.003906 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:08.003921 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:08.003954 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:08.006863 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:08.007602 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:08.007610 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:08.007612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:08.007741 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:08.008122 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:08.008129 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:08.008132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:08.008467 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:08.008822 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:08.009084 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 01:00:08.009309 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 01:00:08.010550 12936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:08.157501 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 12797
I20260809 01:00:08.184732 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 12945
Aug 09 01:00:08 dist-test-slave-07c2 krb5kdc[12526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.82.131@KRBTEST.COM
I20260809 01:00:08.200318 13229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:08.185532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52166 (local address 127.4.82.131:34163)
0809 01:00:08.185955 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:08.185959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:08.185975 (+ 16us) server_negotiation.cc:407] Connection header received
0809 01:00:08.186029 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:08.186033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:08.186090 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:08.186186 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:08.187520 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:08.188212 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:08.189446 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:08.189945 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:08.190043 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:08.194400 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:08.194414 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:08.194430 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:08.194465 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:08.197606 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:08.198233 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:08.198240 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:08.198242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:08.198377 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:08.198707 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:08.198710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:08.198712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:08.199137 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:08.199418 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:08.199773 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 01:00:08.199937 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":213,"threads_started":1}
I20260809 01:00:08.200948 13084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:08.324316 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 12945
I20260809 01:00:08.348345 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 12539
I20260809 01:00:08.349530 12617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:08.463372 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 12539
2026-08-09T01:00:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5891 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:00:08 dist-test-slave-07c2 krb5kdc[13234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:00:08 dist-test-slave-07c2 krb5kdc[13234](info): set up 2 sockets
Aug 09 01:00:08 dist-test-slave-07c2 krb5kdc[13234](info): commencing operation
krb5kdc: starting...
W20260809 01:00:09.601555 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:00:09 dist-test-slave-07c2 krb5kdc[13234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:00:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:00:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:00:09.829237 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41263
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:45501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:41263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:10.150245 13248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:10.150565 13248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:10.150739 13248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:10.157750 13248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:00:10.157886 13248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:10.157991 13248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:10.158097 13248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:00:10.158183 13248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:00:10.167666 13248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:45501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:41263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:10.168880 13248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:10.170147 13248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:10.181629 13253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:10.181687 13254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:10.181588 13256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:10.182762 13248 server_base.cc:1034] running on GCE node
I20260809 01:00:10.183554 13248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:10.184798 13248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:10.185992 13248 hybrid_clock.cc:648] HybridClock initialized: now 1786237210185937 us; error 51 us; skew 500 ppm
I20260809 01:00:10.186317 13248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T01:00:10Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1792 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:00:10 dist-test-slave-07c2 krb5kdc[13265](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:00:10 dist-test-slave-07c2 krb5kdc[13265](info): set up 2 sockets
Aug 09 01:00:10 dist-test-slave-07c2 krb5kdc[13265](info): commencing operation
krb5kdc: starting...
W20260809 01:00:11.378415 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:00:11 dist-test-slave-07c2 krb5kdc[13265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:00:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:00:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:00:11.615444 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:33199
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:33199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:11.937552 13279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:11.937829 13279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:11.938021 13279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:11.945331 13279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:00:11.945452 13279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:11.945539 13279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:11.945627 13279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:00:11.945691 13279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:00:11.955855 13279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:33199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:33199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:11.957126 13279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:11.958396 13279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:11.969941 13287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:11.970027 13285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:11.970206 13284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:11.970970 13279 server_base.cc:1034] running on GCE node
I20260809 01:00:11.971774 13279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:11.973094 13279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:11.974349 13279 hybrid_clock.cc:648] HybridClock initialized: now 1786237211974299 us; error 43 us; skew 500 ppm
I20260809 01:00:11.974697 13279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:11 dist-test-slave-07c2 krb5kdc[13265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237211, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:11.977775 13279 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:00:11.979269 13279 webserver.cc:460] Webserver started at http://127.4.82.190:33795/ using document root <none> and password file <none>
I20260809 01:00:11.980005 13279 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:11.980159 13279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:11.980480 13279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:11.983470 13279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "697c98d83ec64621ad1f104786c8b2c2"
format_stamp: "Formatted at 2026-08-09 01:00:11 on dist-test-slave-07c2"
server_key: "e948d877d94313745ce1be6b3c33d862"
server_key_iv: "f218efa9c725e771cc3d727e962dd5cc"
server_key_version: "encryptionkey@0"
I20260809 01:00:11.984328 13279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "697c98d83ec64621ad1f104786c8b2c2"
format_stamp: "Formatted at 2026-08-09 01:00:11 on dist-test-slave-07c2"
server_key: "e948d877d94313745ce1be6b3c33d862"
server_key_iv: "f218efa9c725e771cc3d727e962dd5cc"
server_key_version: "encryptionkey@0"
I20260809 01:00:11.989279 13279 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 01:00:11.992552 13294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:11.993777 13279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 01:00:11.993961 13279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "697c98d83ec64621ad1f104786c8b2c2"
format_stamp: "Formatted at 2026-08-09 01:00:11 on dist-test-slave-07c2"
server_key: "e948d877d94313745ce1be6b3c33d862"
server_key_iv: "f218efa9c725e771cc3d727e962dd5cc"
server_key_version: "encryptionkey@0"
I20260809 01:00:11.994167 13279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:12.009768 13279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:12.013739 13279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:12.014026 13279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:12.032441 13279 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:33199
I20260809 01:00:12.032478 13349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:33199 every 8 connection(s)
I20260809 01:00:12.033804 13279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:00:12.036309 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 13279
I20260809 01:00:12.036552 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:00:12.037389 4426 external_mini_cluster.cc:1428] Setting key c362f25df369395e76cb94411619f248
I20260809 01:00:12.037977 13350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:12.051960 13350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2: Bootstrap starting.
Aug 09 01:00:12 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:00:12.054977 13350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:12.056354 13350 log.cc:824] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2: Log is configured to *not* fsync() on all Append() calls
I20260809 01:00:12.059631 13350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2: No bootstrap required, opened a new log
I20260809 01:00:12.061658 13353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:12.039498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34058 (local address 127.4.82.190:33199)
0809 01:00:12.040251 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:12.040263 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:12.040302 (+ 39us) server_negotiation.cc:407] Connection header received
0809 01:00:12.042381 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:12.042400 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:12.042763 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:12.043073 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:12.044902 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:12.046113 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:12.047986 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:12.048698 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:12.048829 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:12.053450 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:12.053457 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:12.053470 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:12.053516 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:12.057272 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:12.058048 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:12.058057 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:12.058064 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:12.058268 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:12.058612 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:12.058616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:12.058619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:12.059269 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:12.059588 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:12.059923 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 01:00:12.060365 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":354,"threads_started":1}
I20260809 01:00:12.067472 13350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697c98d83ec64621ad1f104786c8b2c2" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33199 } }
I20260809 01:00:12.067791 13350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:12.067891 13350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697c98d83ec64621ad1f104786c8b2c2, State: Initialized, Role: FOLLOWER
I20260809 01:00:12.068487 13350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697c98d83ec64621ad1f104786c8b2c2" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33199 } }
I20260809 01:00:12.068739 13350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:00:12.068878 13350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:00:12.069057 13350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:12.071118 13350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697c98d83ec64621ad1f104786c8b2c2" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33199 } }
I20260809 01:00:12.071667 13350 leader_election.cc:302] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 697c98d83ec64621ad1f104786c8b2c2; no voters:
I20260809 01:00:12.072242 13350 leader_election.cc:288] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:00:12.072381 13356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:00:12.073153 13356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 1 LEADER]: Becoming Leader. State: Replica: 697c98d83ec64621ad1f104786c8b2c2, State: Running, Role: LEADER
I20260809 01:00:12.073640 13356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697c98d83ec64621ad1f104786c8b2c2" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33199 } }
I20260809 01:00:12.074342 13350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:00:12.076341 13358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 697c98d83ec64621ad1f104786c8b2c2. Latest consensus state: current_term: 1 leader_uuid: "697c98d83ec64621ad1f104786c8b2c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697c98d83ec64621ad1f104786c8b2c2" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33199 } } }
I20260809 01:00:12.076709 13358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:12.078239 13357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "697c98d83ec64621ad1f104786c8b2c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697c98d83ec64621ad1f104786c8b2c2" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33199 } } }
I20260809 01:00:12.078547 13361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:00:12.078548 13357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:12.083884 13361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:00:12.092852 13361 catalog_manager.cc:1347] Generated new cluster ID: 6862336e2e08455cbb9143cca0973225
I20260809 01:00:12.092974 13361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:00:12.110922 13361 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:00:12.112277 13361 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:00:12.121951 13361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2: Generated new TSK 0
I20260809 01:00:12.122778 13361 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 01:00:12.182880 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:33199
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:12.511958 13377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:12.512254 13377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:12.512351 13377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:12.519740 13377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:12.519843 13377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:12.519951 13377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:00:12.529788 13377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:33199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:12.531003 13377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:12.532379 13377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:12.545039 13382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:12.545068 13383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:12.545344 13386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:12.546001 13377 server_base.cc:1034] running on GCE node
I20260809 01:00:12.546597 13377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:12.547580 13377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:12.548804 13377 hybrid_clock.cc:648] HybridClock initialized: now 1786237212548767 us; error 49 us; skew 500 ppm
I20260809 01:00:12.549104 13377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:12 dist-test-slave-07c2 krb5kdc[13265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237212, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:12.557265 13377 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:00:12.558750 13377 webserver.cc:460] Webserver started at http://127.4.82.129:37885/ using document root <none> and password file <none>
I20260809 01:00:12.559425 13377 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:12.559530 13377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:12.559770 13377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:12.562515 13377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "46487790a2f6420e898db3833f3ca4bf"
format_stamp: "Formatted at 2026-08-09 01:00:12 on dist-test-slave-07c2"
server_key: "14198d8624b22379e442c2eb7be0377b"
server_key_iv: "5fbd7c525276b45f899097e1cb720713"
server_key_version: "encryptionkey@0"
I20260809 01:00:12.563230 13377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "46487790a2f6420e898db3833f3ca4bf"
format_stamp: "Formatted at 2026-08-09 01:00:12 on dist-test-slave-07c2"
server_key: "14198d8624b22379e442c2eb7be0377b"
server_key_iv: "5fbd7c525276b45f899097e1cb720713"
server_key_version: "encryptionkey@0"
I20260809 01:00:12.568306 13377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 01:00:12.571509 13394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:12.572845 13377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 01:00:12.573007 13377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "46487790a2f6420e898db3833f3ca4bf"
format_stamp: "Formatted at 2026-08-09 01:00:12 on dist-test-slave-07c2"
server_key: "14198d8624b22379e442c2eb7be0377b"
server_key_iv: "5fbd7c525276b45f899097e1cb720713"
server_key_version: "encryptionkey@0"
I20260809 01:00:12.573180 13377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:12.593775 13377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:12.597543 13377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:12.597872 13377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:12.598841 13377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:12.600131 13377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:12.600204 13377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:12.600396 13377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:12.600458 13377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:12.633941 13377 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:40661
I20260809 01:00:12.633988 13509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:40661 every 8 connection(s)
I20260809 01:00:12.635166 13377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:00:12.636965 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 13377
I20260809 01:00:12.637180 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:00:12.637979 4426 external_mini_cluster.cc:1428] Setting key 3e33a7ac0e980953ce68e8c151ca1d51
Aug 09 01:00:12 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237212, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:00:12.659247 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:12.637727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:60717 (local address 127.4.82.190:33199)
0809 01:00:12.638249 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:12.638255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:12.639306 (+ 1051us) server_negotiation.cc:407] Connection header received
0809 01:00:12.641920 (+ 2614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:12.641924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:12.641984 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:12.642073 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:12.644135 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:12.644849 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:12.647239 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:12.647667 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:12.647766 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:12.652375 (+ 4609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:12.652384 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:12.652392 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:12.652436 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:12.656026 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:12.656737 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:12.656745 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:12.656752 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:12.656912 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:12.657342 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:12.657350 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:12.657356 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:12.657726 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:12.657876 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:12.658525 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 01:00:12.658770 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":257,"threads_started":1}
I20260809 01:00:12.659824 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:12.638270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:60717)
0809 01:00:12.639129 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:12.639178 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:12.641614 (+ 2436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:12.642348 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:12.642360 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:12.643140 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:12.643955 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:12.643981 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:12.645004 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:12.645012 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:12.647103 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:12.647120 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:12.647915 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:12.647925 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:12.648240 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:12.648959 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:12.648998 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:12.652118 (+ 3120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:12.656180 (+ 4062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:12.656187 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:12.656198 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:12.656598 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:12.657025 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:12.657029 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:12.657031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:12.657222 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:12.657842 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:12.657847 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:12.658131 (+ 284us) client_negotiation.cc:769] Sending connection context
0809 01:00:12.658359 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 01:00:12.658708 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":320,"threads_started":1}
I20260809 01:00:12.661695 13510 heartbeater.cc:347] Connected to a master server at 127.4.82.190:33199
I20260809 01:00:12.662001 13510 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:12.662703 13510 heartbeater.cc:511] Master 127.4.82.190:33199 requested a full tablet report, sending...
I20260809 01:00:12.664829 13313 ts_manager.cc:194] Registered new tserver with Master: 46487790a2f6420e898db3833f3ca4bf (127.4.82.129:40661)
I20260809 01:00:12.666235 13313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:60717
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 01:00:12.698107 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:33199
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:13.024257 13516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:13.024501 13516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:13.024688 13516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:13.031988 13516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:13.032110 13516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:13.032279 13516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:00:13.046988 13516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:33199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:13.048347 13516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:13.049932 13516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:13.062826 13522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:13.062904 13521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:13.062879 13524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:13.063812 13516 server_base.cc:1034] running on GCE node
I20260809 01:00:13.064445 13516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:13.065325 13516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:13.066542 13516 hybrid_clock.cc:648] HybridClock initialized: now 1786237213066503 us; error 54 us; skew 500 ppm
I20260809 01:00:13.066851 13516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:13 dist-test-slave-07c2 krb5kdc[13265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237213, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:13.069880 13516 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:00:13.071188 13516 webserver.cc:460] Webserver started at http://127.4.82.130:39873/ using document root <none> and password file <none>
I20260809 01:00:13.071858 13516 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:13.072000 13516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:13.072299 13516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:13.075153 13516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "efa6d10ee93545ceaa91b50241132fed"
format_stamp: "Formatted at 2026-08-09 01:00:13 on dist-test-slave-07c2"
server_key: "58c14e955926ee7cb3d66c2cb1fe3cae"
server_key_iv: "ad0a3458af118f5c2aa194f62215b538"
server_key_version: "encryptionkey@0"
I20260809 01:00:13.076071 13516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "efa6d10ee93545ceaa91b50241132fed"
format_stamp: "Formatted at 2026-08-09 01:00:13 on dist-test-slave-07c2"
server_key: "58c14e955926ee7cb3d66c2cb1fe3cae"
server_key_iv: "ad0a3458af118f5c2aa194f62215b538"
server_key_version: "encryptionkey@0"
I20260809 01:00:13.081245 13516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 01:00:13.084513 13532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:13.085763 13516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 01:00:13.085927 13516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "efa6d10ee93545ceaa91b50241132fed"
format_stamp: "Formatted at 2026-08-09 01:00:13 on dist-test-slave-07c2"
server_key: "58c14e955926ee7cb3d66c2cb1fe3cae"
server_key_iv: "ad0a3458af118f5c2aa194f62215b538"
server_key_version: "encryptionkey@0"
I20260809 01:00:13.086112 13516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:13.101039 13516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:13.104674 13516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:13.104960 13516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:13.105845 13516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:13.107115 13516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:13.107219 13516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:13.107394 13516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:13.107471 13516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:13.140833 13516 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:40143
I20260809 01:00:13.140913 13652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:40143 every 8 connection(s)
I20260809 01:00:13.142077 13516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:00:13.150177 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 13516
I20260809 01:00:13.150347 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:00:13.150804 4426 external_mini_cluster.cc:1428] Setting key 72eb64bf730cc45699fc46069bd41684
Aug 09 01:00:13 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237213, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:00:13.161854 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:13.144548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:59559 (local address 127.4.82.190:33199)
0809 01:00:13.144785 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:13.144791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:13.145880 (+ 1089us) server_negotiation.cc:407] Connection header received
0809 01:00:13.147594 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:13.147598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:13.147668 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:13.147759 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:13.149111 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:13.149790 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:13.151547 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:13.152064 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:13.152165 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:13.156079 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:13.156084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:13.156087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:13.156117 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:13.158647 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:13.159607 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:13.159611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:13.159614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:13.159740 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:13.160207 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:13.160210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:13.160219 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:13.160451 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:13.160617 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:13.161291 (+ 674us) server_negotiation.cc:299] Negotiation successful
0809 01:00:13.161511 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 01:00:13.162755 13655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:13.145026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:59559)
0809 01:00:13.145716 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:13.145753 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:13.147359 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:13.147949 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:13.147956 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:13.148429 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:13.148968 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:13.148985 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:13.149960 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:13.149964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:13.151402 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:13.151415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:13.152276 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:13.152284 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:13.152548 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:13.153209 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:13.153232 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:13.155872 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:13.158784 (+ 2912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:13.158795 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:13.158811 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:13.159487 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:13.159849 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:13.159856 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:13.159862 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:13.160096 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:13.160591 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:13.160598 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:13.160903 (+ 305us) client_negotiation.cc:769] Sending connection context
0809 01:00:13.161173 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 01:00:13.161634 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":274,"threads_started":1}
I20260809 01:00:13.164196 13653 heartbeater.cc:347] Connected to a master server at 127.4.82.190:33199
I20260809 01:00:13.164490 13653 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:13.165302 13653 heartbeater.cc:511] Master 127.4.82.190:33199 requested a full tablet report, sending...
I20260809 01:00:13.166785 13313 ts_manager.cc:194] Registered new tserver with Master: efa6d10ee93545ceaa91b50241132fed (127.4.82.130:40143)
I20260809 01:00:13.167634 13313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:59559
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 01:00:13.204528 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:33199
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:13.531484 13658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:13.531786 13658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:13.531958 13658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:13.539202 13658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:13.539395 13658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:13.539575 13658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:00:13.549351 13658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:33199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:13.550627 13658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:13.551939 13658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:13.564915 13664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:13.564831 13667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:13.564880 13665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:13.565668 13658 server_base.cc:1034] running on GCE node
I20260809 01:00:13.566286 13658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:13.567233 13658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:13.568475 13658 hybrid_clock.cc:648] HybridClock initialized: now 1786237213568429 us; error 44 us; skew 500 ppm
I20260809 01:00:13.568800 13658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:13 dist-test-slave-07c2 krb5kdc[13265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237213, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:13.572317 13658 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:00:13.574316 13658 webserver.cc:460] Webserver started at http://127.4.82.131:42581/ using document root <none> and password file <none>
I20260809 01:00:13.575381 13658 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:13.575628 13658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:13.576035 13658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:13.580148 13658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "c789c1427d114567bd82bff52bd68a68"
format_stamp: "Formatted at 2026-08-09 01:00:13 on dist-test-slave-07c2"
server_key: "dae4ed61e7e611041b0b649349fbb920"
server_key_iv: "c4480c5470508426cf26a269d508c04c"
server_key_version: "encryptionkey@0"
I20260809 01:00:13.581284 13658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "c789c1427d114567bd82bff52bd68a68"
format_stamp: "Formatted at 2026-08-09 01:00:13 on dist-test-slave-07c2"
server_key: "dae4ed61e7e611041b0b649349fbb920"
server_key_iv: "c4480c5470508426cf26a269d508c04c"
server_key_version: "encryptionkey@0"
I20260809 01:00:13.587020 13658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 01:00:13.590639 13675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:13.592090 13658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:00:13.592278 13658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "c789c1427d114567bd82bff52bd68a68"
format_stamp: "Formatted at 2026-08-09 01:00:13 on dist-test-slave-07c2"
server_key: "dae4ed61e7e611041b0b649349fbb920"
server_key_iv: "c4480c5470508426cf26a269d508c04c"
server_key_version: "encryptionkey@0"
I20260809 01:00:13.592485 13658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:13.609051 13658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:13.613356 13658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:13.613648 13658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:13.614599 13658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:13.615917 13658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:13.616034 13658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:13.616187 13658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:13.616266 13658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:13.651816 13658 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:45353
I20260809 01:00:13.651868 13793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:45353 every 8 connection(s)
I20260809 01:00:13.653038 13658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:00:13.658461 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 13658
I20260809 01:00:13.658643 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:00:13.659271 4426 external_mini_cluster.cc:1428] Setting key f0cec74bcdcc3b2e31214eb963d1930a
Aug 09 01:00:13 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237213, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:00:13.669230 13510 heartbeater.cc:503] Master 127.4.82.190:33199 was elected leader, sending a full tablet report...
I20260809 01:00:13.673787 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:13.655567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:46513 (local address 127.4.82.190:33199)
0809 01:00:13.655754 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:13.655758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:13.656799 (+ 1041us) server_negotiation.cc:407] Connection header received
0809 01:00:13.658879 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:13.658884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:13.658954 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:13.659041 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:13.661018 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:13.662143 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:13.663828 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:13.664293 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:13.664390 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:13.667684 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:13.667689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:13.667692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:13.667725 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:13.670789 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:13.671546 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:13.671556 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:13.671558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:13.671685 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:13.672118 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:13.672127 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:13.672129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:13.672385 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:13.672543 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:13.673235 (+ 692us) server_negotiation.cc:299] Negotiation successful
0809 01:00:13.673449 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 01:00:13.674911 13797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:13.655948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:46513)
0809 01:00:13.656639 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:13.656677 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:13.658576 (+ 1899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:13.659344 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:13.659356 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:13.660022 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:13.660828 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:13.660856 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:13.662308 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:13.662312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:13.663700 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:13.663712 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:13.664482 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:13.664487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:13.664677 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:13.665310 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:13.665332 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:13.667501 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:13.670918 (+ 3417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:13.670926 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:13.670940 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:13.671422 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:13.671798 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:13.671804 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:13.671807 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:13.671994 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:13.672514 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:13.672522 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:13.672849 (+ 327us) client_negotiation.cc:769] Sending connection context
0809 01:00:13.673108 (+ 259us) client_negotiation.cc:240] Negotiation successful
0809 01:00:13.673597 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":276,"threads_started":1}
I20260809 01:00:13.676254 13794 heartbeater.cc:347] Connected to a master server at 127.4.82.190:33199
I20260809 01:00:13.676539 13794 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:13.677299 13794 heartbeater.cc:511] Master 127.4.82.190:33199 requested a full tablet report, sending...
I20260809 01:00:13.678612 13313 ts_manager.cc:194] Registered new tserver with Master: c789c1427d114567bd82bff52bd68a68 (127.4.82.131:45353)
I20260809 01:00:13.679455 13313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:46513
I20260809 01:00:13.686528 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab.
Entry for principal oryx/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab.
WARNING: no policy specified for oryx/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab.
Entry for principal oryx/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab.
WARNING: no policy specified for oryx/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab.
Entry for principal oryx/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab.
I20260809 01:00:13.830299 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 13377
Aug 09 01:00:13 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 01:00:13.846984 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:13.831613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32792 (local address 127.4.82.129:40661)
0809 01:00:13.832309 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:13.832323 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:13.832348 (+ 25us) server_negotiation.cc:407] Connection header received
0809 01:00:13.832407 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:13.832413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:13.832618 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:13.832794 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:13.834235 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:13.835262 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:13.836584 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:13.837160 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:13.837297 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:13.841415 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:13.841426 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:13.841439 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:13.841483 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:13.844194 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:13.844864 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:13.844870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:13.844872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:13.845011 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:13.845404 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:13.845409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:13.845412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:13.845757 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:13.846063 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:13.846365 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 01:00:13.846650 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":336,"threads_started":1}
I20260809 01:00:13.848315 13505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:13.932838 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 13377
I20260809 01:00:13.951328 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 13516
Aug 09 01:00:13 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 01:00:13.966840 13807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:13.952319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.4.82.130:40143)
0809 01:00:13.952924 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:13.952931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:13.952957 (+ 26us) server_negotiation.cc:407] Connection header received
0809 01:00:13.953020 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:13.953026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:13.953230 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:13.953366 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:13.954571 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:13.955622 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:13.956906 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:13.957419 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:13.957514 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:13.961925 (+ 4411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:13.961935 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:13.961948 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:13.961980 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:13.964605 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:13.965188 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:13.965194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:13.965196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:13.965309 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:13.965598 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:13.965601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:13.965603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:13.965907 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:13.966128 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:13.966366 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 01:00:13.966549 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":314,"threads_started":1}
I20260809 01:00:13.968151 13648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:14.050572 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 13516
I20260809 01:00:14.068171 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 13658
Aug 09 01:00:14 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 01:00:14.085064 13809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:14.069183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45514 (local address 127.4.82.131:45353)
0809 01:00:14.069825 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:14.069832 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:14.069861 (+ 29us) server_negotiation.cc:407] Connection header received
0809 01:00:14.069924 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:14.069931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:14.070132 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:14.070309 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:14.071706 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:14.072734 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:14.074041 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:14.074664 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:14.074808 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:14.079393 (+ 4585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:14.079406 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:14.079420 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:14.079455 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:14.082534 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:14.083149 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:14.083155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:14.083158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:14.083419 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:14.083791 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:14.083795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:14.083797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:14.084121 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:14.084356 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:14.084584 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 01:00:14.084784 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":336,"threads_started":1}
I20260809 01:00:14.086306 13789 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:14.168553 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 13658
I20260809 01:00:14.187263 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 13279
I20260809 01:00:14.188431 13345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:14.291962 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 13279
I20260809 01:00:14.314564 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:33199
--webserver_interface=127.4.82.190
--webserver_port=33795
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:33199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:14.643020 13812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:14.643293 13812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:14.643399 13812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:14.650440 13812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:00:14.650534 13812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:14.650580 13812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:14.650667 13812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:00:14.650755 13812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:00:14.659883 13812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:33199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:33199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=33795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:14.661099 13812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:14.662359 13812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:14.674021 13817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:14.674058 13818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:14.673977 13820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:14.674973 13812 server_base.cc:1034] running on GCE node
I20260809 01:00:14.675818 13812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:14.677111 13812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:14.678289 13812 hybrid_clock.cc:648] HybridClock initialized: now 1786237214678254 us; error 31 us; skew 500 ppm
I20260809 01:00:14.678643 13812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:14 dist-test-slave-07c2 krb5kdc[13265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237214, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:14.681808 13812 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:00:14.683151 13812 webserver.cc:460] Webserver started at http://127.4.82.190:33795/ using document root <none> and password file <none>
I20260809 01:00:14.683857 13812 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:14.684012 13812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:14.689153 13812 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 01:00:14.691850 13827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:14.693066 13812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:00:14.693245 13812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "697c98d83ec64621ad1f104786c8b2c2"
format_stamp: "Formatted at 2026-08-09 01:00:11 on dist-test-slave-07c2"
server_key: "e948d877d94313745ce1be6b3c33d862"
server_key_iv: "f218efa9c725e771cc3d727e962dd5cc"
server_key_version: "encryptionkey@0"
I20260809 01:00:14.694168 13812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:14.712929 13812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:14.716898 13812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:14.717154 13812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:14.735433 13812 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:33199
I20260809 01:00:14.735466 13880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:33199 every 8 connection(s)
I20260809 01:00:14.736850 13812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:00:14.739594 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 13812
I20260809 01:00:14.740420 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:40661
--local_ip_for_outbound_sockets=127.4.82.129
--tserver_master_addrs=127.4.82.190:33199
--webserver_port=37885
--webserver_interface=127.4.82.129
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.82.129
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:00:14.743534 13881 sys_catalog.cc:263] Verifying existing consensus state
I20260809 01:00:14.746582 13881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2: Bootstrap starting.
I20260809 01:00:14.764613 13881 log.cc:824] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2: Log is configured to *not* fsync() on all Append() calls
I20260809 01:00:14.772881 13881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 01:00:14.773384 13881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2: Bootstrap complete.
I20260809 01:00:14.782110 13881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697c98d83ec64621ad1f104786c8b2c2" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33199 } }
I20260809 01:00:14.782920 13881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697c98d83ec64621ad1f104786c8b2c2, State: Initialized, Role: FOLLOWER
I20260809 01:00:14.783535 13881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697c98d83ec64621ad1f104786c8b2c2" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33199 } }
I20260809 01:00:14.783795 13881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:00:14.783921 13881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:00:14.784116 13881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:00:14.787783 13881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697c98d83ec64621ad1f104786c8b2c2" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33199 } }
I20260809 01:00:14.788267 13881 leader_election.cc:302] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 697c98d83ec64621ad1f104786c8b2c2; no voters:
I20260809 01:00:14.788859 13881 leader_election.cc:288] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 01:00:14.789057 13886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 01:00:14.790264 13886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [term 2 LEADER]: Becoming Leader. State: Replica: 697c98d83ec64621ad1f104786c8b2c2, State: Running, Role: LEADER
I20260809 01:00:14.790946 13881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:00:14.790980 13886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697c98d83ec64621ad1f104786c8b2c2" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33199 } }
I20260809 01:00:14.794111 13889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 697c98d83ec64621ad1f104786c8b2c2. Latest consensus state: current_term: 2 leader_uuid: "697c98d83ec64621ad1f104786c8b2c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697c98d83ec64621ad1f104786c8b2c2" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33199 } } }
I20260809 01:00:14.794502 13887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "697c98d83ec64621ad1f104786c8b2c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "697c98d83ec64621ad1f104786c8b2c2" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33199 } } }
I20260809 01:00:14.794739 13889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:14.794862 13887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:14.795748 13896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:00:14.808696 13896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:00:14.810539 13896 catalog_manager.cc:1259] Loaded cluster ID: 6862336e2e08455cbb9143cca0973225
I20260809 01:00:14.810751 13896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:00:14.815991 13896 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:00:14.817883 13896 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 697c98d83ec64621ad1f104786c8b2c2: Loaded TSK: 0
I20260809 01:00:14.819805 13896 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 01:00:15.108389 13884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:15.108706 13884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:15.108868 13884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:15.115919 13884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:15.116077 13884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:15.116245 13884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:00:15.125670 13884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.129.keytab
--principal=oryx/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:40661
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=37885
--tserver_master_addrs=127.4.82.190:33199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:15.126897 13884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:15.128219 13884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:15.142880 13911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:15.143006 13912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:15.143739 13914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:15.149488 13884 server_base.cc:1034] running on GCE node
I20260809 01:00:15.150138 13884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:15.151077 13884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:15.152308 13884 hybrid_clock.cc:648] HybridClock initialized: now 1786237215152251 us; error 49 us; skew 500 ppm
I20260809 01:00:15.152611 13884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237215, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:15.155741 13884 init.cc:377] Logged in from keytab as oryx/127.4.82.129@KRBTEST.COM (short username oryx)
I20260809 01:00:15.157253 13884 webserver.cc:460] Webserver started at http://127.4.82.129:37885/ using document root <none> and password file <none>
I20260809 01:00:15.157923 13884 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:15.158072 13884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:15.163451 13884 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 01:00:15.166229 13921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:15.167748 13884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:00:15.167918 13884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "46487790a2f6420e898db3833f3ca4bf"
format_stamp: "Formatted at 2026-08-09 01:00:12 on dist-test-slave-07c2"
server_key: "14198d8624b22379e442c2eb7be0377b"
server_key_iv: "5fbd7c525276b45f899097e1cb720713"
server_key_version: "encryptionkey@0"
I20260809 01:00:15.168895 13884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:15.202240 13884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:15.206611 13884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:15.206908 13884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:15.207904 13884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:15.209208 13884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:15.209342 13884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:15.209468 13884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:15.209561 13884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:15.242326 13884 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:40661
I20260809 01:00:15.242415 14040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:40661 every 8 connection(s)
I20260809 01:00:15.243821 13884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:00:15.245889 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 13884
I20260809 01:00:15.246599 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:40143
--local_ip_for_outbound_sockets=127.4.82.130
--tserver_master_addrs=127.4.82.190:33199
--webserver_port=39873
--webserver_interface=127.4.82.130
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.82.130
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:15.263758 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:15.246815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:38335)
0809 01:00:15.248225 (+ 1410us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:15.248283 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:15.250960 (+ 2677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:15.252300 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:15.252312 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:15.253066 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:15.253848 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:15.253876 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.255479 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.255487 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:15.257138 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:15.257150 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.258593 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.258598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:15.258775 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:15.259424 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:15.259444 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:15.262824 (+ 3380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1110,"thread_start_us":879,"threads_started":1}
W20260809 01:00:15.264422 14041 heartbeater.cc:644] Failed to heartbeat to 127.4.82.190:33199 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.82.190:33199: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
W20260809 01:00:15.264444 14043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:15.246796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:38335 (local address 127.4.82.190:33199)
0809 01:00:15.247588 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:15.247606 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:15.248427 (+ 821us) server_negotiation.cc:407] Connection header received
0809 01:00:15.251545 (+ 3118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:15.251556 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:15.251841 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:15.252110 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:15.254042 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.255254 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.257286 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.258406 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.258586 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:15.263266 (+ 4680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:38335: BlockingRecv error: recv got EOF from 127.4.82.129:38335 (error 108)
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":286,"threads_started":1}
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:15.272748 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:15.264970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:34361)
0809 01:00:15.265220 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:15.265239 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:15.265330 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:15.265612 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:15.265617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:15.265930 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:15.266052 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:15.266059 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.266986 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.266991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:15.267933 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:15.267944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.268651 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.268657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:15.268850 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:15.269410 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:15.269426 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:15.272426 (+ 3000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120}
W20260809 01:00:15.272774 14043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:15.265108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:34361 (local address 127.4.82.190:33199)
0809 01:00:15.265314 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:15.265319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:15.265337 (+ 18us) server_negotiation.cc:407] Connection header received
0809 01:00:15.265426 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:15.265430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:15.265493 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:15.265588 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:15.266197 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.266854 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.268077 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.268462 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.268629 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:15.272570 (+ 3941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:34361: BlockingRecv error: recv got EOF from 127.4.82.129:34361 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:15.281424 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:15.273588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:53307)
0809 01:00:15.273769 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:15.273787 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:15.273881 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:15.274310 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:15.274313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:15.274560 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:15.274701 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:15.274708 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.276116 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.276119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:15.277005 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:15.277015 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.277914 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.277918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:15.278038 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:15.278594 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:15.278610 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:15.281237 (+ 2627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 01:00:15.281703 14041 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 01:00:15.281853 14043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:15.273780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:53307 (local address 127.4.82.190:33199)
0809 01:00:15.273972 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:15.273981 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:15.274003 (+ 22us) server_negotiation.cc:407] Connection header received
0809 01:00:15.274063 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:15.274071 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:15.274175 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:15.274289 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:15.274843 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.275986 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.277151 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.277786 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.277907 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:15.281490 (+ 3583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:53307: BlockingRecv error: recv got EOF from 127.4.82.129:53307 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 01:00:15.579100 14045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:15.579530 14045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:15.579664 14045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:15.586668 14045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:15.586776 14045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:15.586884 14045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:00:15.596603 14045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.130.keytab
--principal=oryx/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:40143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=39873
--tserver_master_addrs=127.4.82.190:33199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:15.597922 14045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:15.599462 14045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:15.612104 14055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:15.612179 14051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:15.612151 14052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:15.612990 14045 server_base.cc:1034] running on GCE node
I20260809 01:00:15.613571 14045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:15.614436 14045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:15.615653 14045 hybrid_clock.cc:648] HybridClock initialized: now 1786237215615598 us; error 46 us; skew 500 ppm
I20260809 01:00:15.615949 14045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237215, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:15.619029 14045 init.cc:377] Logged in from keytab as oryx/127.4.82.130@KRBTEST.COM (short username oryx)
I20260809 01:00:15.620445 14045 webserver.cc:460] Webserver started at http://127.4.82.130:39873/ using document root <none> and password file <none>
I20260809 01:00:15.621064 14045 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:15.621176 14045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:15.626130 14045 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 01:00:15.628650 14063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:15.629972 14045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:00:15.630105 14045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "efa6d10ee93545ceaa91b50241132fed"
format_stamp: "Formatted at 2026-08-09 01:00:13 on dist-test-slave-07c2"
server_key: "58c14e955926ee7cb3d66c2cb1fe3cae"
server_key_iv: "ad0a3458af118f5c2aa194f62215b538"
server_key_version: "encryptionkey@0"
I20260809 01:00:15.630995 14045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:15.648969 14045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:15.653175 14045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:15.653395 14045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:15.654282 14045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:15.655560 14045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:15.655645 14045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:15.655733 14045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:15.655787 14045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:15.689050 14045 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:40143
I20260809 01:00:15.689141 14181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:40143 every 8 connection(s)
I20260809 01:00:15.690493 14045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:00:15.693347 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 14045
I20260809 01:00:15.693948 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:45353
--local_ip_for_outbound_sockets=127.4.82.131
--tserver_master_addrs=127.4.82.190:33199
--webserver_port=42581
--webserver_interface=127.4.82.131
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.82.131
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 01:00:15.707935 14043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:15.693228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:40211 (local address 127.4.82.190:33199)
0809 01:00:15.693465 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:15.693472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:15.694700 (+ 1228us) server_negotiation.cc:407] Connection header received
0809 01:00:15.696566 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:15.696575 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:15.696665 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:15.696795 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:15.698398 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.699580 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.701109 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.701845 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.701984 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:15.707618 (+ 5634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:40211: BlockingRecv error: recv got EOF from 127.4.82.130:40211 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 01:00:15.708389 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:15.693711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:40211)
0809 01:00:15.694511 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:15.694563 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:15.696304 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:15.697061 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:15.697069 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:15.697574 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:15.698205 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:15.698237 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.699758 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.699761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:15.700960 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:15.700972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.702080 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.702088 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:15.702401 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:15.703171 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:15.703201 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:15.707400 (+ 4199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":354,"threads_started":1}
W20260809 01:00:15.709125 14182 heartbeater.cc:644] Failed to heartbeat to 127.4.82.190:33199 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.82.190:33199: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:15.717602 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:15.709708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:47973)
0809 01:00:15.709883 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:15.709903 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:15.709999 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:15.710324 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:15.710329 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:15.710659 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:15.710789 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:15.710815 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.711858 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.711864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:15.712788 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:15.712798 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.713491 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.713496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:15.713668 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:15.714299 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:15.714315 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:15.717308 (+ 2993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 01:00:15.718055 14043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:15.709826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:47973 (local address 127.4.82.190:33199)
0809 01:00:15.710002 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:15.710007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:15.710025 (+ 18us) server_negotiation.cc:407] Connection header received
0809 01:00:15.710104 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:15.710109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:15.710180 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:15.710285 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:15.710946 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.711726 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.712933 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.713341 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.713479 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:15.717521 (+ 4042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:47973: BlockingRecv error: recv got EOF from 127.4.82.130:47973 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:15 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:15.726712 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:15.718261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:48863)
0809 01:00:15.718423 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:15.718440 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:15.718537 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:15.719137 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:15.719140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:15.719474 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:15.719612 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:15.719619 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.721140 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.721144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:15.722029 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:15.722039 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.723071 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.723075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:15.723214 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:15.723832 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:15.723849 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:15.726512 (+ 2663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 01:00:15.726974 14043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:15.718579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:48863 (local address 127.4.82.190:33199)
0809 01:00:15.718801 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:15.718809 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:15.718832 (+ 23us) server_negotiation.cc:407] Connection header received
0809 01:00:15.718891 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:15.718899 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:15.718985 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:15.719100 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:15.719751 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.720996 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.722180 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:15.722933 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:15.723082 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:15.726764 (+ 3682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:48863: BlockingRecv error: recv got EOF from 127.4.82.130:48863 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 01:00:15.727104 14182 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 01:00:16.027446 14185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:16.027712 14185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:16.027880 14185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:16.035039 14185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:16.035163 14185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:16.035405 14185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:00:16.044687 14185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/krb5kdc/oryx_127.4.82.131.keytab
--principal=oryx/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:45353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=42581
--tserver_master_addrs=127.4.82.190:33199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:16.045885 14185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:16.047129 14185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:16.060153 14190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:16.060209 14193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:16.060089 14191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:16.060920 14185 server_base.cc:1034] running on GCE node
I20260809 01:00:16.061455 14185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:16.062314 14185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:16.063531 14185 hybrid_clock.cc:648] HybridClock initialized: now 1786237216063492 us; error 54 us; skew 500 ppm
I20260809 01:00:16.063830 14185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:16 dist-test-slave-07c2 krb5kdc[13265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237216, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:16.066833 14185 init.cc:377] Logged in from keytab as oryx/127.4.82.131@KRBTEST.COM (short username oryx)
I20260809 01:00:16.068181 14185 webserver.cc:460] Webserver started at http://127.4.82.131:42581/ using document root <none> and password file <none>
I20260809 01:00:16.068955 14185 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:16.069125 14185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:16.074764 14185 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 01:00:16.077311 14202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:16.078505 14185 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 01:00:16.078675 14185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "c789c1427d114567bd82bff52bd68a68"
format_stamp: "Formatted at 2026-08-09 01:00:13 on dist-test-slave-07c2"
server_key: "dae4ed61e7e611041b0b649349fbb920"
server_key_iv: "c4480c5470508426cf26a269d508c04c"
server_key_version: "encryptionkey@0"
I20260809 01:00:16.079699 14185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:16.099578 14185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:16.103348 14185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:16.103615 14185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:16.104498 14185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:16.105787 14185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:16.105924 14185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:16.106060 14185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:16.106151 14185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:16.139756 14185 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:45353
I20260809 01:00:16.139832 14322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:45353 every 8 connection(s)
I20260809 01:00:16.141026 14185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:00:16.141392 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 14185
Aug 09 01:00:16 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:16 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:16.157734 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:16.142466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34072 (local address 127.4.82.190:33199)
0809 01:00:16.142715 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:16.142723 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:16.143315 (+ 592us) server_negotiation.cc:407] Connection header received
0809 01:00:16.143504 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:16.143508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:16.143573 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:16.143668 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:16.145599 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.146357 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.147640 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.148223 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.148355 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:16.151377 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:16.151386 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:16.151398 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:16.151445 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:16.154592 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:16.155438 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:16.155444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:16.155449 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:16.155602 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:16.156032 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:16.156035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:16.156037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:16.156420 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:16.156755 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:16.157139 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 01:00:16.157440 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 01:00:16.159364 14327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:16.145268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:35227)
0809 01:00:16.146028 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:16.146069 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:16.147771 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:16.148463 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:16.148474 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:16.149200 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:16.149970 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:16.149998 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.151071 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.151076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:16.152370 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:16.152381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.153474 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.153479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:16.153671 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:16.154247 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:16.154273 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:16.158373 (+ 4100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":326,"threads_started":1}
W20260809 01:00:16.159825 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:16.144796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:35227 (local address 127.4.82.190:33199)
0809 01:00:16.145298 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:16.145304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:16.146184 (+ 880us) server_negotiation.cc:407] Connection header received
0809 01:00:16.148007 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:16.148012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:16.148099 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:16.148207 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:16.150162 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.150904 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.152535 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.153261 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.153397 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:16.158675 (+ 5278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:35227: BlockingRecv error: recv got EOF from 127.4.82.131:35227 (error 108)
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":260,"threads_started":1}
W20260809 01:00:16.160198 14323 heartbeater.cc:644] Failed to heartbeat to 127.4.82.190:33199 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.82.190:33199: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Aug 09 01:00:16 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:16 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:16.167657 14327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:16.160749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:48407)
0809 01:00:16.160909 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:16.160925 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:16.161025 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:16.161347 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:16.161369 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:16.161690 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:16.161825 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:16.161833 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.162763 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.162766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:16.163649 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:16.163659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.164305 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.164309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:16.164424 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:16.164886 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:16.164902 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:16.167460 (+ 2558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 01:00:16.167819 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:16.160898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:48407 (local address 127.4.82.190:33199)
0809 01:00:16.161070 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:16.161079 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:16.161098 (+ 19us) server_negotiation.cc:407] Connection header received
0809 01:00:16.161144 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:16.161149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:16.161211 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:16.161314 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:16.161943 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.162644 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.163792 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.164189 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.164279 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:16.167632 (+ 3353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:48407: BlockingRecv error: recv got EOF from 127.4.82.131:48407 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 01:00:16 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:16 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:16.175734 14327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:16.168282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:47097)
0809 01:00:16.168449 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:16.168463 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:16.168533 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:16.168805 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:16.168808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:16.169055 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:16.169200 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:16.169207 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.170102 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.170106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:16.170953 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:16.170963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.171661 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.171666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:16.171875 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:16.172449 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:16.172471 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:16.175511 (+ 3040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 01:00:16.175909 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:16.168389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:47097 (local address 127.4.82.190:33199)
0809 01:00:16.168595 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:16.168598 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:16.168612 (+ 14us) server_negotiation.cc:407] Connection header received
0809 01:00:16.168647 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:16.168650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:16.168695 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:16.168769 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:16.169310 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.169990 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.171063 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.171533 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.171654 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:16.175698 (+ 4044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:47097: BlockingRecv error: recv got EOF from 127.4.82.131:47097 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
W20260809 01:00:16.176055 14323 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 01:00:16 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:16 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:16.290616 14328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:16.282253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:59123)
0809 01:00:16.282798 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:16.282814 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:16.282945 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:16.283245 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:16.283249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:16.283816 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:16.284006 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:16.284015 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.284940 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.284944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:16.285935 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:16.285946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.286614 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.286618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:16.286740 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:16.287400 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:16.287418 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:16.290290 (+ 2872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":337,"threads_started":1}
W20260809 01:00:16.290815 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:16.282354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:59123 (local address 127.4.82.190:33199)
0809 01:00:16.282524 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:16.282529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:16.282889 (+ 360us) server_negotiation.cc:407] Connection header received
0809 01:00:16.283037 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:16.283041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:16.283109 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:16.283188 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:16.284129 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.284817 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.286065 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.286453 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.286548 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:16.290517 (+ 3969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:59123: BlockingRecv error: recv got EOF from 127.4.82.129:59123 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 01:00:16 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:16 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:16.736107 14329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:16.727785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:52089)
0809 01:00:16.728361 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:16.728381 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:16.728508 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:16.728803 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:16.728806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:16.729242 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:16.729425 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:16.729433 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.730358 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.730361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:16.731318 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:16.731328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.732132 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.732137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:16.732269 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:16.732801 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:16.732817 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:16.735812 (+ 2995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":302,"threads_started":1}
W20260809 01:00:16.736189 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:16.727883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:52089 (local address 127.4.82.190:33199)
0809 01:00:16.728128 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:16.728134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:16.728477 (+ 343us) server_negotiation.cc:407] Connection header received
0809 01:00:16.728621 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:16.728625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:16.728688 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:16.728764 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:16.729546 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.730245 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.731430 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:16.731896 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:16.732012 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:16.735975 (+ 3963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:52089: BlockingRecv error: recv got EOF from 127.4.82.130:52089 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 01:00:17 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:17 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:17.185019 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:17.176740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:53049)
0809 01:00:17.177296 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:17.177313 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:17.177427 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:17.177699 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:17.177702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:17.178182 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:17.178376 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:17.178384 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:17.179348 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:17.179352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:17.180316 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:17.180327 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:17.181030 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:17.181034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:17.181156 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:17.181784 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:17.181802 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:17.184726 (+ 2924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":299,"threads_started":1}
W20260809 01:00:17.185225 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:17.176828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:53049 (local address 127.4.82.190:33199)
0809 01:00:17.177010 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:17.177014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:17.177388 (+ 374us) server_negotiation.cc:407] Connection header received
0809 01:00:17.177509 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:17.177512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:17.177570 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:17.177649 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:17.178497 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:17.179203 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:17.180432 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:17.180885 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:17.180973 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:17.184951 (+ 3978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:53049: BlockingRecv error: recv got EOF from 127.4.82.131:53049 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 01:00:17 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:17 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:17.300048 14331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:17.291670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:47041)
0809 01:00:17.292238 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:17.292259 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:17.292428 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:17.292851 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:17.292856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:17.293304 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:17.293454 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:17.293463 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:17.294405 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:17.294408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:17.295334 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:17.295345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:17.295979 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:17.295983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:17.296092 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:17.296668 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:17.296686 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:17.299812 (+ 3126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":301,"threads_started":1}
W20260809 01:00:17.300266 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:17.291712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:47041 (local address 127.4.82.190:33199)
0809 01:00:17.291899 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:17.291907 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:17.292361 (+ 454us) server_negotiation.cc:407] Connection header received
0809 01:00:17.292588 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:17.292593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:17.292696 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:17.292797 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:17.293577 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:17.294287 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:17.295452 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:17.295854 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:17.295945 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:17.300011 (+ 4066us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:47041: BlockingRecv error: recv got EOF from 127.4.82.129:47041 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 01:00:17 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:17 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:17.750808 14332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:17.737136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:46413)
0809 01:00:17.737713 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:17.737730 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:17.737841 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:17.738187 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:17.738190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:17.738582 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:17.738739 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:17.738748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:17.740197 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:17.740205 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:17.741359 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:17.741372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:17.742387 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:17.742393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:17.742549 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:17.747749 (+ 5200us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:17.747768 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:17.750545 (+ 2777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":315,"threads_started":1}
W20260809 01:00:17.750952 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:17.737225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:46413 (local address 127.4.82.190:33199)
0809 01:00:17.737434 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:17.737440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:17.737814 (+ 374us) server_negotiation.cc:407] Connection header received
0809 01:00:17.737953 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:17.737958 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:17.738042 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:17.738160 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:17.738885 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:17.740039 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:17.741497 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:17.742179 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:17.742343 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:17.750752 (+ 8409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:46413: BlockingRecv error: recv got EOF from 127.4.82.130:46413 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 01:00:18 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:18 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:18.194653 14333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:18.185937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:60959)
0809 01:00:18.186491 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:18.186507 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:18.186663 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:18.187026 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:18.187030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:18.187493 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:18.187633 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:18.187642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:18.188688 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:18.188693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:18.189595 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:18.189605 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:18.190313 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:18.190319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:18.190488 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:18.191194 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:18.191213 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:18.194416 (+ 3203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":292,"threads_started":1}
W20260809 01:00:18.194856 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:18.186130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:60959 (local address 127.4.82.190:33199)
0809 01:00:18.186361 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:18.186367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:18.186592 (+ 225us) server_negotiation.cc:407] Connection header received
0809 01:00:18.186800 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:18.186806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:18.186887 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:18.186989 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:18.187778 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:18.188529 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:18.189743 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:18.190151 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:18.190302 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:18.194611 (+ 4309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:60959: BlockingRecv error: recv got EOF from 127.4.82.131:60959 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 09 01:00:18 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:18 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:18.309319 14334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:18.301071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:58761)
0809 01:00:18.301558 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:18.301574 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:18.301732 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:18.302100 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:18.302105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:18.302510 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:18.302680 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:18.302689 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:18.303785 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:18.303792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:18.304690 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:18.304701 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:18.305380 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:18.305386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:18.305560 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:18.306173 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:18.306189 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:18.308926 (+ 2737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":275,"threads_started":1}
W20260809 01:00:18.309430 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:18.301121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:58761 (local address 127.4.82.190:33199)
0809 01:00:18.301343 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:18.301351 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:18.301666 (+ 315us) server_negotiation.cc:407] Connection header received
0809 01:00:18.301901 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:18.301905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:18.301963 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:18.302063 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:18.302834 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:18.303645 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:18.304834 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:18.305228 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:18.305357 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:18.309171 (+ 3814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:58761: BlockingRecv error: recv got EOF from 127.4.82.129:58761 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 01:00:18 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:18 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 01:00:18.760391 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:18.751939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:45279 (local address 127.4.82.190:33199)
0809 01:00:18.752115 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:18.752119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:18.752437 (+ 318us) server_negotiation.cc:407] Connection header received
0809 01:00:18.752565 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:18.752568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:18.752625 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:18.752704 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:18.753380 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:18.754095 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:18.755258 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:18.755703 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:18.755816 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:18.760165 (+ 4349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:45279: BlockingRecv error: recv got EOF from 127.4.82.130:45279 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 01:00:18.760377 14335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:18.751840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:45279)
0809 01:00:18.752351 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:18.752367 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:18.752474 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:18.752754 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:18.752757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:18.753087 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:18.753258 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:18.753266 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:18.754221 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:18.754225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:18.755127 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:18.755138 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:18.755846 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:18.755850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:18.755976 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:18.756698 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:18.756724 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:18.759985 (+ 3261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":309,"threads_started":1}
Aug 09 01:00:19 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:19 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:19.204367 14336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:19.195596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:45709)
0809 01:00:19.196134 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:19.196149 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:19.196271 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:19.196654 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:19.196660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:19.197171 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:19.197398 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:19.197413 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:19.198668 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:19.198673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:19.199751 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:19.199765 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:19.200405 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:19.200409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:19.200524 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:19.201162 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:19.201180 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:19.204108 (+ 2928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":301,"threads_started":1}
W20260809 01:00:19.204530 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:19.195688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:45709 (local address 127.4.82.190:33199)
0809 01:00:19.195883 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:19.195888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:19.196258 (+ 370us) server_negotiation.cc:407] Connection header received
0809 01:00:19.196382 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:19.196387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:19.196484 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:19.196609 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:19.197574 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:19.198507 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:19.199872 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:19.200286 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:19.200382 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:19.204295 (+ 3913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:45709: BlockingRecv error: recv got EOF from 127.4.82.131:45709 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 01:00:19 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:19 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:19.319032 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:19.310557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:38679)
0809 01:00:19.311073 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:19.311088 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:19.311259 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:19.311616 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:19.311621 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:19.312047 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:19.312206 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:19.312216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:19.313251 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:19.313258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:19.314212 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:19.314224 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:19.315191 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:19.315197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:19.315435 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:19.316056 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:19.316074 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:19.318809 (+ 2735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":298,"threads_started":1}
W20260809 01:00:19.319236 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:19.310632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:38679 (local address 127.4.82.190:33199)
0809 01:00:19.310870 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:19.310877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:19.311180 (+ 303us) server_negotiation.cc:407] Connection header received
0809 01:00:19.311409 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:19.311421 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:19.311484 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:19.311584 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:19.312351 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:19.313108 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:19.314356 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:19.315011 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:19.315170 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:19.319006 (+ 3836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:38679: BlockingRecv error: recv got EOF from 127.4.82.129:38679 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 09 01:00:19 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:19 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:19.769549 14338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:19.761342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:48601)
0809 01:00:19.761828 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:19.761844 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:19.761956 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:19.762188 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:19.762191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:19.762557 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:19.762709 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:19.762717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:19.763734 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:19.763738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:19.764718 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:19.764729 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:19.765681 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:19.765687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:19.765913 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:19.766493 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:19.766509 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:19.769327 (+ 2818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":271,"threads_started":1}
W20260809 01:00:19.769704 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:19.761481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:48601 (local address 127.4.82.190:33199)
0809 01:00:19.761730 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:19.761736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:19.761918 (+ 182us) server_negotiation.cc:407] Connection header received
0809 01:00:19.762036 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:19.762040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:19.762099 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:19.762175 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:19.762825 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:19.763623 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:19.764846 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:19.765508 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:19.765639 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:19.769503 (+ 3864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:48601: BlockingRecv error: recv got EOF from 127.4.82.130:48601 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 01:00:20 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:20 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:20.213789 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:20.205320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:47291)
0809 01:00:20.205820 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:20.205836 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:20.205945 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:20.206250 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:20.206253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:20.206619 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:20.206785 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:20.206792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:20.207750 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:20.207754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:20.208806 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:20.208822 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:20.209679 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:20.209686 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:20.209885 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:20.210607 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:20.210624 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:20.213554 (+ 2930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":272,"threads_started":1}
W20260809 01:00:20.213977 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:20.205337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:47291 (local address 127.4.82.190:33199)
0809 01:00:20.205509 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:20.205513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:20.205911 (+ 398us) server_negotiation.cc:407] Connection header received
0809 01:00:20.206041 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:20.206045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:20.206122 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:20.206202 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:20.206911 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:20.207640 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:20.208969 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:20.209495 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:20.209612 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:20.213719 (+ 4107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:47291: BlockingRecv error: recv got EOF from 127.4.82.131:47291 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 01:00:20 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:20 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:20.328402 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:20.320100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:39995)
0809 01:00:20.320573 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:20.320602 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:20.320787 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:20.321114 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:20.321119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:20.321560 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:20.321718 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:20.321726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:20.322725 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:20.322730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:20.323694 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:20.323705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:20.324392 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:20.324397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:20.324568 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:20.325113 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:20.325129 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:20.328169 (+ 3040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":238,"threads_started":1}
W20260809 01:00:20.328655 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:20.320130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:39995 (local address 127.4.82.190:33199)
0809 01:00:20.320356 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:20.320363 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:20.320717 (+ 354us) server_negotiation.cc:407] Connection header received
0809 01:00:20.320919 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:20.320928 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:20.320986 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:20.321084 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:20.321862 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:20.322571 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:20.323836 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:20.324237 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:20.324369 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:20.328377 (+ 4008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:39995: BlockingRecv error: recv got EOF from 127.4.82.129:39995 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 01:00:20 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:20 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:20.779990 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:20.770620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:55591)
0809 01:00:20.771213 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:20.771231 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:20.771413 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:20.771837 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:20.771842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:20.772566 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:20.772801 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:20.772815 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:20.774327 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:20.774335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:20.775498 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:20.775511 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:20.776182 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:20.776186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:20.776327 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:20.776870 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:20.776887 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:20.779744 (+ 2857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":290,"threads_started":1}
W20260809 01:00:20.780170 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:20.770878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:55591 (local address 127.4.82.190:33199)
0809 01:00:20.771086 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:20.771093 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:20.771322 (+ 229us) server_negotiation.cc:407] Connection header received
0809 01:00:20.771547 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:20.771552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:20.771644 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:20.771752 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:20.772967 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:20.774153 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:20.775630 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:20.776055 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:20.776165 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:20.779944 (+ 3779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:55591: BlockingRecv error: recv got EOF from 127.4.82.130:55591 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 01:00:21 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:21 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:21.223127 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:21.214748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:60641)
0809 01:00:21.215215 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:21.215231 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:21.215412 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:21.215774 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:21.215778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:21.216253 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:21.216420 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:21.216429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:21.217455 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:21.217463 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:21.218369 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:21.218380 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:21.219074 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:21.219080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:21.219246 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:21.219856 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:21.219873 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:21.222873 (+ 3000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":209,"threads_started":1}
W20260809 01:00:21.223326 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:21.214774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:60641 (local address 127.4.82.190:33199)
0809 01:00:21.214974 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:21.214983 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:21.215332 (+ 349us) server_negotiation.cc:407] Connection header received
0809 01:00:21.215534 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:21.215546 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:21.215625 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:21.215730 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:21.216589 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:21.217312 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:21.218506 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:21.218920 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:21.219052 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:21.223061 (+ 4009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:60641: BlockingRecv error: recv got EOF from 127.4.82.131:60641 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 01:00:21 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:21 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:21.336966 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:21.329408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:60367)
0809 01:00:21.329858 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:21.329874 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:21.329985 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:21.330272 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:21.330275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:21.330653 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:21.330819 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:21.330827 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:21.331915 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:21.331919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:21.332803 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:21.332814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:21.333460 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:21.333464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:21.333581 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:21.334133 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:21.334150 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:21.336758 (+ 2608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":247,"threads_started":1}
W20260809 01:00:21.337159 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:21.329496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:60367 (local address 127.4.82.190:33199)
0809 01:00:21.329754 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:21.329760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:21.329948 (+ 188us) server_negotiation.cc:407] Connection header received
0809 01:00:21.330084 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:21.330088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:21.330148 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:21.330230 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:21.330933 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:21.331778 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:21.332927 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:21.333333 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:21.333424 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:21.336936 (+ 3512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:60367: BlockingRecv error: recv got EOF from 127.4.82.129:60367 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 01:00:21 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:21 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:21.789281 14344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:21.781059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:47527)
0809 01:00:21.781546 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:21.781562 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:21.781699 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:21.781980 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:21.781983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:21.782372 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:21.782549 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:21.782558 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:21.783535 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:21.783539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:21.784485 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:21.784497 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:21.785152 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:21.785156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:21.785274 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:21.785832 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:21.785848 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:21.788950 (+ 3102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":281,"threads_started":1}
W20260809 01:00:21.789383 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:21.781164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:47527 (local address 127.4.82.190:33199)
0809 01:00:21.781423 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:21.781429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:21.781668 (+ 239us) server_negotiation.cc:407] Connection header received
0809 01:00:21.781800 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:21.781803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:21.781862 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:21.781941 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:21.782692 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:21.783415 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:21.784611 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:21.785016 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:21.785106 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:21.789152 (+ 4046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:47527: BlockingRecv error: recv got EOF from 127.4.82.130:47527 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 09 01:00:22 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:22 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:22.232281 14345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:22.224187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:49053)
0809 01:00:22.224693 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:22.224709 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:22.224824 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:22.225111 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:22.225114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:22.225517 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:22.225663 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:22.225671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:22.226763 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:22.226767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:22.227676 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:22.227687 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:22.228372 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:22.228376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:22.228495 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:22.229169 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:22.229187 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:22.232062 (+ 2875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":296,"threads_started":1}
W20260809 01:00:22.232546 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:22.224301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:49053 (local address 127.4.82.190:33199)
0809 01:00:22.224555 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:22.224570 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:22.224786 (+ 216us) server_negotiation.cc:407] Connection header received
0809 01:00:22.224908 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:22.224911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:22.224985 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:22.225070 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:22.225811 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:22.226630 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:22.227806 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:22.228238 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:22.228337 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:22.232258 (+ 3921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:49053: BlockingRecv error: recv got EOF from 127.4.82.131:49053 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 01:00:22 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:22 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:22.346612 14347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:22.337920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:33275)
0809 01:00:22.338536 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:22.338571 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:22.338736 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:22.339075 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:22.339080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:22.339601 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:22.339771 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:22.339779 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:22.340786 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:22.340794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:22.341707 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:22.341725 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:22.342594 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:22.342602 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:22.342792 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:22.343473 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:22.343490 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:22.346382 (+ 2892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":279,"threads_started":1}
W20260809 01:00:22.346829 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:22.337978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:33275 (local address 127.4.82.190:33199)
0809 01:00:22.338215 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:22.338224 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:22.338669 (+ 445us) server_negotiation.cc:407] Connection header received
0809 01:00:22.338863 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:22.338868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:22.338940 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:22.339042 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:22.339914 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:22.340637 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:22.341857 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:22.342289 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:22.342541 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:22.346586 (+ 4045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:33275: BlockingRecv error: recv got EOF from 127.4.82.129:33275 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 01:00:22 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:22 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:22.803853 14348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:22.790293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:57777)
0809 01:00:22.790896 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:22.790914 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:22.791046 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:22.791387 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:22.791391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:22.791902 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:22.792069 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:22.792077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:22.793001 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:22.793005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:22.793917 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:22.793928 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:22.794613 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:22.794617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:22.794732 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:22.800761 (+ 6029us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:22.800779 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:22.803589 (+ 2810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":286,"threads_started":1}
W20260809 01:00:22.803992 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:22.790344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:57777 (local address 127.4.82.190:33199)
0809 01:00:22.790543 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:22.790547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:22.791007 (+ 460us) server_negotiation.cc:407] Connection header received
0809 01:00:22.791143 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:22.791147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:22.791206 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:22.791310 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:22.792193 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:22.792873 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:22.794037 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:22.794489 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:22.794580 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:22.803784 (+ 9204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:57777: BlockingRecv error: recv got EOF from 127.4.82.130:57777 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 01:00:23 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:23 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:23.241818 14349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:23.233120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:40625)
0809 01:00:23.233730 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:23.233747 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:23.233862 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:23.234271 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:23.234276 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:23.234945 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:23.235191 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:23.235203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:23.236363 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:23.236368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:23.237295 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:23.237306 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:23.238038 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:23.238045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:23.238213 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:23.238781 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:23.238798 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:23.241561 (+ 2763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":297,"threads_started":1}
W20260809 01:00:23.241979 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:23.233227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:40625 (local address 127.4.82.190:33199)
0809 01:00:23.233454 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:23.233463 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:23.233845 (+ 382us) server_negotiation.cc:407] Connection header received
0809 01:00:23.234001 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:23.234004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:23.234083 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:23.234204 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:23.235407 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:23.236214 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:23.237450 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:23.237883 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:23.238021 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:23.241762 (+ 3741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:40625: BlockingRecv error: recv got EOF from 127.4.82.131:40625 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 01:00:23 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:23 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:23.356310 14350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:23.347606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:60881)
0809 01:00:23.348128 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:23.348144 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:23.348303 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:23.348659 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:23.348664 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:23.349095 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:23.349260 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:23.349269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:23.350306 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:23.350312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:23.351713 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:23.351728 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:23.352739 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:23.352744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:23.352861 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:23.353413 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:23.353430 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:23.356080 (+ 2650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":260,"threads_started":1}
W20260809 01:00:23.356531 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:23.347724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:60881 (local address 127.4.82.190:33199)
0809 01:00:23.347939 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:23.347945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:23.348226 (+ 281us) server_negotiation.cc:407] Connection header received
0809 01:00:23.348438 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:23.348444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:23.348516 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:23.348627 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:23.349401 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:23.350159 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:23.351897 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:23.352568 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:23.352670 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:23.356301 (+ 3631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:60881: BlockingRecv error: recv got EOF from 127.4.82.129:60881 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 01:00:23 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:23 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:23.813391 14351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:23.804830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:34241)
0809 01:00:23.805434 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:23.805450 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:23.805563 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:23.805866 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:23.805869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:23.806254 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:23.806411 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:23.806418 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:23.807567 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:23.807571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:23.808487 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:23.808499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:23.809194 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:23.809198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:23.809314 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:23.809980 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:23.809998 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:23.813128 (+ 3130us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":317,"threads_started":1}
W20260809 01:00:23.813638 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:23.804942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:34241 (local address 127.4.82.190:33199)
0809 01:00:23.805109 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:23.805114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:23.805523 (+ 409us) server_negotiation.cc:407] Connection header received
0809 01:00:23.805656 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:23.805660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:23.805738 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:23.805819 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:23.806532 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:23.807432 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:23.808616 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:23.809038 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:23.809137 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:23.813360 (+ 4223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:34241: BlockingRecv error: recv got EOF from 127.4.82.130:34241 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 01:00:24 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:24 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:24.251857 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:24.242943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:37731)
0809 01:00:24.243920 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:24.243941 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:24.244102 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:24.244452 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:24.244456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:24.244910 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:24.245058 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:24.245067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:24.246073 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:24.246078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:24.247110 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:24.247122 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:24.248021 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:24.248027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:24.248206 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:24.248799 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:24.248816 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:24.251616 (+ 2800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":444,"threads_started":1}
W20260809 01:00:24.252045 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:24.242975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:37731 (local address 127.4.82.190:33199)
0809 01:00:24.243177 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:24.243186 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:24.244034 (+ 848us) server_negotiation.cc:407] Connection header received
0809 01:00:24.244249 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:24.244252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:24.244312 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:24.244416 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:24.245214 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:24.245932 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:24.247260 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:24.247831 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:24.247975 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:24.251820 (+ 3845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:37731: BlockingRecv error: recv got EOF from 127.4.82.131:37731 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 01:00:24 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:24 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:24.365892 14353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:24.357265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:32905)
0809 01:00:24.357781 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:24.357798 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:24.357917 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:24.358207 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:24.358210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:24.358600 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:24.358773 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:24.358781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:24.359743 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:24.359746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:24.360689 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:24.360700 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:24.361509 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:24.361516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:24.361713 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:24.362452 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:24.362474 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:24.365535 (+ 3061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":281,"threads_started":1}
W20260809 01:00:24.365978 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:24.357340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:32905 (local address 127.4.82.190:33199)
0809 01:00:24.357595 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:24.357602 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:24.357884 (+ 282us) server_negotiation.cc:407] Connection header received
0809 01:00:24.358023 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:24.358027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:24.358087 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:24.358167 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:24.358899 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:24.359625 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:24.360836 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:24.361346 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:24.361447 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:24.365739 (+ 4292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:32905: BlockingRecv error: recv got EOF from 127.4.82.129:32905 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 01:00:24 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:24 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:24.822705 14354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:24.814285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:41717)
0809 01:00:24.814788 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:24.814804 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:24.814919 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:24.815170 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:24.815173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:24.815592 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:24.815785 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:24.815794 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:24.816822 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:24.816825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:24.817760 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:24.817771 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:24.818450 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:24.818454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:24.818572 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:24.819132 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:24.819148 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:24.822334 (+ 3186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":275,"threads_started":1}
W20260809 01:00:24.822827 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:24.814381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:41717 (local address 127.4.82.190:33199)
0809 01:00:24.814646 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:24.814651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:24.814885 (+ 234us) server_negotiation.cc:407] Connection header received
0809 01:00:24.815015 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:24.815018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:24.815080 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:24.815157 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:24.815904 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:24.816713 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:24.817893 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:24.818318 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:24.818407 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:24.822541 (+ 4134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:41717: BlockingRecv error: recv got EOF from 127.4.82.130:41717 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 09 01:00:25 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:25 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:25.261772 14355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:25.252836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:52875)
0809 01:00:25.253340 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:25.253355 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:25.253466 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:25.253783 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:25.253787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:25.254210 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:25.254379 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:25.254387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:25.255430 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:25.255434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:25.256447 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:25.256460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:25.257476 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:25.257482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:25.257674 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:25.258199 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:25.258215 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:25.261526 (+ 3311us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":270,"threads_started":1}
W20260809 01:00:25.261938 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:25.252940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:52875 (local address 127.4.82.190:33199)
0809 01:00:25.253204 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:25.253210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:25.253434 (+ 224us) server_negotiation.cc:407] Connection header received
0809 01:00:25.253557 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:25.253560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:25.253619 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:25.253696 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:25.254518 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:25.255225 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:25.256620 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:25.257299 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:25.257439 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:25.261729 (+ 4290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:52875: BlockingRecv error: recv got EOF from 127.4.82.131:52875 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 09 01:00:25 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:25 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:25.375264 14356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:25.366992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:60779)
0809 01:00:25.367757 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:25.367782 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:25.367923 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:25.368273 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:25.368276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:25.368667 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:25.368813 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:25.368821 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:25.369776 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:25.369779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:25.370724 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:25.370735 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:25.371425 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:25.371429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:25.371547 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:25.372231 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:25.372248 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:25.375013 (+ 2765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":540,"threads_started":1}
W20260809 01:00:25.375485 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:25.367062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:60779 (local address 127.4.82.190:33199)
0809 01:00:25.367248 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:25.367254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:25.367844 (+ 590us) server_negotiation.cc:407] Connection header received
0809 01:00:25.368058 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:25.368068 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:25.368138 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:25.368221 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:25.368934 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:25.369647 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:25.370842 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:25.371273 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:25.371389 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:25.375223 (+ 3834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:60779: BlockingRecv error: recv got EOF from 127.4.82.129:60779 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 01:00:25 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:25 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:25.832458 14357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:25.823712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:57977)
0809 01:00:25.824243 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:25.824269 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:25.824374 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:25.824663 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:25.824666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:25.825041 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:25.825183 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:25.825190 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:25.826381 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:25.826387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:25.827479 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:25.827491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:25.828234 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:25.828238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:25.828355 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:25.828997 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:25.829014 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:25.832112 (+ 3098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":317,"threads_started":1}
W20260809 01:00:25.832571 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:25.823811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:57977 (local address 127.4.82.190:33199)
0809 01:00:25.823993 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:25.823997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:25.824342 (+ 345us) server_negotiation.cc:407] Connection header received
0809 01:00:25.824471 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:25.824475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:25.824533 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:25.824613 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:25.825483 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:25.826213 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:25.827597 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:25.828102 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:25.828208 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:25.832303 (+ 4095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:57977: BlockingRecv error: recv got EOF from 127.4.82.130:57977 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 01:00:26 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:26 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:26.270694 14358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:26.262696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:58695)
0809 01:00:26.263251 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:26.263268 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:26.263412 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:26.263707 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:26.263711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:26.264075 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:26.264217 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:26.264225 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:26.265184 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:26.265187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:26.266111 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:26.266121 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:26.266804 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:26.266809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:26.266939 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:26.267539 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:26.267556 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:26.270362 (+ 2806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":308,"threads_started":1}
W20260809 01:00:26.270836 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:26.262830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:58695 (local address 127.4.82.190:33199)
0809 01:00:26.263043 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:26.263047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:26.263379 (+ 332us) server_negotiation.cc:407] Connection header received
0809 01:00:26.263510 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:26.263514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:26.263572 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:26.263650 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:26.264348 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:26.265076 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:26.266231 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:26.266632 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:26.266734 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:26.270554 (+ 3820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:58695: BlockingRecv error: recv got EOF from 127.4.82.131:58695 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 09 01:00:26 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:26 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:26.384606 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:26.376111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:60147)
0809 01:00:26.376724 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:26.376745 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:26.376892 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:26.377308 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:26.377313 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:26.377880 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:26.378123 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:26.378132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:26.379094 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:26.379098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:26.379971 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:26.379981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:26.380643 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:26.380647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:26.380755 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:26.381315 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:26.381332 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:26.384286 (+ 2954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":373,"threads_started":1}
W20260809 01:00:26.384668 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:26.376230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:60147 (local address 127.4.82.190:33199)
0809 01:00:26.376559 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:26.376565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:26.376851 (+ 286us) server_negotiation.cc:407] Connection header received
0809 01:00:26.377062 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:26.377068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:26.377149 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:26.377247 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:26.378260 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:26.378992 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:26.380081 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:26.380520 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:26.380608 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:26.384499 (+ 3891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:60147: BlockingRecv error: recv got EOF from 127.4.82.129:60147 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
Aug 09 01:00:26 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:26 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:26.841423 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:26.833482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:38325)
0809 01:00:26.834088 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:26.834107 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:26.834246 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:26.834565 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:26.834569 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:26.834999 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:26.835181 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:26.835190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:26.836235 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:26.836239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:26.837109 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:26.837119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:26.837779 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:26.837783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:26.837896 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:26.838453 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:26.838469 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:26.841190 (+ 2721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":320,"threads_started":1}
W20260809 01:00:26.841574 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:26.833567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:38325 (local address 127.4.82.190:33199)
0809 01:00:26.833844 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:26.833853 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:26.834199 (+ 346us) server_negotiation.cc:407] Connection header received
0809 01:00:26.834368 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:26.834371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:26.834433 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:26.834512 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:26.835351 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:26.836125 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:26.837238 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:26.837657 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:26.837753 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:26.841358 (+ 3605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:38325: BlockingRecv error: recv got EOF from 127.4.82.130:38325 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
Aug 09 01:00:27 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:27 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:27.280529 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:27.271777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:32837)
0809 01:00:27.272366 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:27.272382 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:27.272542 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:27.272903 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:27.272908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:27.273478 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:27.273667 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:27.273676 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:27.274654 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:27.274660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:27.275603 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:27.275615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:27.276363 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:27.276384 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:27.276571 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:27.277198 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:27.277215 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:27.280297 (+ 3082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":325,"threads_started":1}
W20260809 01:00:27.280705 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:27.271866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:32837 (local address 127.4.82.190:33199)
0809 01:00:27.272124 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:27.272131 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:27.272476 (+ 345us) server_negotiation.cc:407] Connection header received
0809 01:00:27.272705 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:27.272708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:27.272767 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:27.272869 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:27.273815 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:27.274515 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:27.275752 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:27.276168 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:27.276338 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:27.280454 (+ 4116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:32837: BlockingRecv error: recv got EOF from 127.4.82.131:32837 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 01:00:27 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:27 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:27.394088 14362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:27.385534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:41085)
0809 01:00:27.386157 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:27.386179 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:27.386321 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:27.386703 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:27.386708 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:27.387103 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:27.387260 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:27.387268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:27.388340 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:27.388347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:27.389301 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:27.389312 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:27.390064 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:27.390069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:27.390254 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:27.390803 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:27.390820 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:27.393867 (+ 3047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":359,"threads_started":1}
W20260809 01:00:27.394351 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:27.385576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:41085 (local address 127.4.82.190:33199)
0809 01:00:27.385761 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:27.385769 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:27.386257 (+ 488us) server_negotiation.cc:407] Connection header received
0809 01:00:27.386463 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:27.386467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:27.386527 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:27.386635 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:27.387434 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:27.388201 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:27.389447 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:27.389925 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:27.390054 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:27.394051 (+ 3997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:41085: BlockingRecv error: recv got EOF from 127.4.82.129:41085 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 01:00:27 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:27 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:27.867630 14363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:27.842489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:35879)
0809 01:00:27.843102 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:27.843117 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:27.843229 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:27.843618 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:27.843623 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:27.844086 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:27.844256 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:27.844265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:27.845230 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:27.845234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:27.846338 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:27.846354 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:27.846984 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:27.846988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:27.847116 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:27.864186 (+ 17070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:27.864205 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:27.867398 (+ 3193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":350,"threads_started":1}
W20260809 01:00:27.867817 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:27.842541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:35879 (local address 127.4.82.190:33199)
0809 01:00:27.842771 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:27.842781 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:27.843182 (+ 401us) server_negotiation.cc:407] Connection header received
0809 01:00:27.843359 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:27.843364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:27.843451 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:27.843558 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:27.844378 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:27.845106 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:27.846471 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:27.846844 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:27.846932 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:27.867597 (+ 20665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:35879: BlockingRecv error: recv got EOF from 127.4.82.130:35879 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 01:00:28 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:28 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:28.289824 14364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:28.281373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:38931)
0809 01:00:28.281935 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:28.281951 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:28.282063 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:28.282391 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:28.282394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:28.282786 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:28.282982 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:28.282996 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:28.284182 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:28.284186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:28.285086 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:28.285096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:28.285721 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:28.285725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:28.285847 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:28.286617 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:28.286646 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:28.289590 (+ 2944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":289,"threads_started":1}
W20260809 01:00:28.290026 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:28.281429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:38931 (local address 127.4.82.190:33199)
0809 01:00:28.281587 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:28.281592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:28.282019 (+ 427us) server_negotiation.cc:407] Connection header received
0809 01:00:28.282154 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:28.282167 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:28.282226 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:28.282353 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:28.283173 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:28.284051 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:28.285207 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:28.285591 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:28.285685 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:28.289791 (+ 4106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:38931: BlockingRecv error: recv got EOF from 127.4.82.131:38931 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 01:00:28 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:28 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:28.403445 14365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:28.394932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:49069)
0809 01:00:28.395490 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:28.395506 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:28.395673 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:28.396009 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:28.396015 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:28.396440 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:28.396613 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:28.396621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:28.397719 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:28.397723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:28.398734 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:28.398746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:28.399592 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:28.399599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:28.399770 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:28.400289 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:28.400306 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:28.403060 (+ 2754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":271,"threads_started":1}
W20260809 01:00:28.403544 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:28.395020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:49069 (local address 127.4.82.190:33199)
0809 01:00:28.395254 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:28.395263 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:28.395589 (+ 326us) server_negotiation.cc:407] Connection header received
0809 01:00:28.395799 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:28.395804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:28.395876 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:28.395975 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:28.396769 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:28.397534 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:28.398876 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:28.399392 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:28.399517 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:28.403267 (+ 3750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:49069: BlockingRecv error: recv got EOF from 127.4.82.129:49069 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 01:00:28 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:28 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:28.876959 14366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:28.868755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:51727)
0809 01:00:28.869323 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:28.869339 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:28.869449 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:28.869730 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:28.869733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:28.870125 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:28.870297 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:28.870306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:28.871363 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:28.871375 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:28.872434 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:28.872447 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:28.873236 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:28.873243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:28.873380 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:28.873963 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:28.873981 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:28.876687 (+ 2706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":290,"threads_started":1}
W20260809 01:00:28.877228 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:28.868862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:51727 (local address 127.4.82.190:33199)
0809 01:00:28.869063 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:28.869067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:28.869413 (+ 346us) server_negotiation.cc:407] Connection header received
0809 01:00:28.869545 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:28.869548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:28.869608 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:28.869687 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:28.870419 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:28.871151 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:28.872567 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:28.873034 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:28.873122 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:28.876957 (+ 3835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:51727: BlockingRecv error: recv got EOF from 127.4.82.130:51727 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 01:00:29 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:29 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:29.299227 14367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:29.290660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:57843)
0809 01:00:29.291501 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:29.291517 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:29.291628 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:29.291954 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:29.291958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:29.292335 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:29.292488 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:29.292496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:29.293462 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:29.293465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:29.294372 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:29.294383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:29.295163 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:29.295173 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:29.295384 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:29.295975 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:29.295993 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:29.298931 (+ 2938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":550,"threads_started":1}
W20260809 01:00:29.299410 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:29.290768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:57843 (local address 127.4.82.190:33199)
0809 01:00:29.290999 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:29.291045 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:29.291601 (+ 556us) server_negotiation.cc:407] Connection header received
0809 01:00:29.291719 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:29.291723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:29.291787 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:29.291927 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:29.292626 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:29.293345 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:29.294525 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:29.294972 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:29.295075 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:29.299153 (+ 4078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:57843: BlockingRecv error: recv got EOF from 127.4.82.131:57843 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 01:00:29 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:29 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:29.412606 14368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:29.404438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:49475)
0809 01:00:29.404930 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:29.404946 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:29.405056 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:29.405341 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:29.405344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:29.405693 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:29.405866 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:29.405874 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:29.407071 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:29.407077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:29.408153 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:29.408166 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:29.408871 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:29.408875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:29.408990 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:29.409545 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:29.409561 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:29.412371 (+ 2810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":263,"threads_started":1}
W20260809 01:00:29.412863 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:29.404533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:49475 (local address 127.4.82.190:33199)
0809 01:00:29.404818 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:29.404822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:29.405016 (+ 194us) server_negotiation.cc:407] Connection header received
0809 01:00:29.405155 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:29.405160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:29.405221 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:29.405301 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:29.405993 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:29.406896 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:29.408281 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:29.408752 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:29.408847 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:29.412561 (+ 3714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:49475: BlockingRecv error: recv got EOF from 127.4.82.129:49475 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
Aug 09 01:00:29 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:29 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:29.886408 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:29.877926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:38127)
0809 01:00:29.878424 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:29.878444 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:29.878609 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:29.879033 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:29.879037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:29.879506 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:29.879683 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:29.879692 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:29.880807 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:29.880811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:29.881722 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:29.881733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:29.882356 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:29.882360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:29.882476 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:29.883247 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:29.883264 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:29.886153 (+ 2889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":249,"threads_started":1}
W20260809 01:00:29.886564 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:29.877989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:38127 (local address 127.4.82.190:33199)
0809 01:00:29.878247 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:29.878253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:29.878550 (+ 297us) server_negotiation.cc:407] Connection header received
0809 01:00:29.878754 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:29.878759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:29.878877 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:29.878989 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:29.879847 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:29.880636 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:29.881845 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:29.882232 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:29.882328 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:29.886339 (+ 4011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:38127: BlockingRecv error: recv got EOF from 127.4.82.130:38127 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 09 01:00:30 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:30 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.131@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:30.308598 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:30.300165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.131:52389)
0809 01:00:30.300662 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:30.300678 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:30.300793 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:30.301145 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:30.301149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:30.301519 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:30.301722 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:30.301737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:30.303022 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:30.303026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:30.303966 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:30.303977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:30.304648 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:30.304652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:30.304771 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:30.305443 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:30.305461 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:30.308332 (+ 2871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":277,"threads_started":1}
W20260809 01:00:30.308840 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:30.300241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:52389 (local address 127.4.82.190:33199)
0809 01:00:30.300425 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:30.300429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:30.300757 (+ 328us) server_negotiation.cc:407] Connection header received
0809 01:00:30.300918 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:30.300921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:30.300981 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:30.301060 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:30.301846 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:30.302883 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:30.304102 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:30.304523 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:30.304617 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:30.308562 (+ 3945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.131:52389: BlockingRecv error: recv got EOF from 127.4.82.131:52389 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 01:00:30 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:30 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.129@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:30.422302 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:30.413997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.129:37225)
0809 01:00:30.414584 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:30.414601 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:30.414712 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:30.414995 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:30.414998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:30.415432 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:30.415620 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:30.415628 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:30.416579 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:30.416583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:30.417541 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:30.417551 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:30.418455 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:30.418479 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:30.418682 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:30.419379 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:30.419397 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:30.422054 (+ 2657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":334,"threads_started":1}
W20260809 01:00:30.422436 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:30.414061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:37225 (local address 127.4.82.190:33199)
0809 01:00:30.414267 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:30.414271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:30.414680 (+ 409us) server_negotiation.cc:407] Connection header received
0809 01:00:30.414812 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:30.414816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:30.414871 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:30.414952 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:30.415744 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:30.416461 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:30.417696 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:30.418318 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:30.418442 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:30.422236 (+ 3794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.129:37225: BlockingRecv error: recv got EOF from 127.4.82.129:37225 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 01:00:30 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 01:00:30 dist-test-slave-07c2 krb5kdc[13265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.82.130@KRBTEST.COM for oryx/127.4.82.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 01:00:30.895941 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:30.887570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33199 (local address 127.4.82.130:36311)
0809 01:00:30.888097 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:30.888112 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:30.888218 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:30.888491 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:30.888494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:30.888917 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:30.889105 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:30.889113 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:30.890253 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:30.890259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:30.891489 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:30.891502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:30.892197 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:30.892203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:30.892382 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:30.892929 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:30.892945 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:30.895678 (+ 2733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.82.190:33199: Server oryx/127.4.82.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":295,"threads_started":1}
W20260809 01:00:30.896126 14326 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:30.887641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:36311 (local address 127.4.82.190:33199)
0809 01:00:30.887799 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:30.887804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:30.888187 (+ 383us) server_negotiation.cc:407] Connection header received
0809 01:00:30.888310 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:30.888314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:30.888372 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:30.888451 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:30.889223 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:30.890104 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:30.891630 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:30.892048 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:30.892192 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:30.895880 (+ 3688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.82.130:36311: BlockingRecv error: recv got EOF from 127.4.82.130:36311 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 01:00:31.148697 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 13884
I20260809 01:00:31.160467 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:31.149717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46464 (local address 127.4.82.129:40661)
0809 01:00:31.150309 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:31.150316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:31.150342 (+ 26us) server_negotiation.cc:407] Connection header received
0809 01:00:31.150399 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:31.150406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:31.150627 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:31.150801 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:31.152259 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:31.153356 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:31.154503 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:31.155061 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:31.155156 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:31.157589 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:31.157597 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:31.157609 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:31.157645 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:31.160011 (+ 2366us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.82.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 01:00:31.160262 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46464: Request ticket server kudu/127.4.82.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":314,"threads_started":1}
W20260809 01:00:31.160593 14374 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46464: Request ticket server kudu/127.4.82.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 01:00:31.160620 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 13884: Not authorized: Client connection negotiation failed: client connection to 127.4.82.129:40661: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.82.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 01:00:31.160846 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 13884
I20260809 01:00:31.179633 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 14045
I20260809 01:00:31.191675 14375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:31.180635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45312 (local address 127.4.82.130:40143)
0809 01:00:31.181180 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:31.181200 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:31.181230 (+ 30us) server_negotiation.cc:407] Connection header received
0809 01:00:31.181287 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:31.181294 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:31.181504 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:31.181699 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:31.183069 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:31.184156 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:31.185405 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:31.185947 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:31.186089 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:31.188623 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:31.188632 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:31.188644 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:31.188689 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:31.190985 (+ 2296us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.82.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 01:00:31.191331 (+ 346us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45312: Request ticket server kudu/127.4.82.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":258,"threads_started":1}
W20260809 01:00:31.191745 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 14045: Not authorized: Client connection negotiation failed: client connection to 127.4.82.130:40143: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.82.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 01:00:31.191790 14375 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45312: Request ticket server kudu/127.4.82.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 01:00:31.191880 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 14045
I20260809 01:00:31.212257 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 14185
I20260809 01:00:31.224152 14376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:31.213261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49186 (local address 127.4.82.131:45353)
0809 01:00:31.213907 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:31.213914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:31.213940 (+ 26us) server_negotiation.cc:407] Connection header received
0809 01:00:31.214003 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:31.214009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:31.214230 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:31.214386 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:31.215750 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:31.216862 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:31.218034 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:31.218561 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:31.218653 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:31.221179 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:31.221187 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:31.221199 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:31.221241 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:31.223632 (+ 2391us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.82.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 01:00:31.223877 (+ 245us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49186: Request ticket server kudu/127.4.82.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":357,"threads_started":1}
W20260809 01:00:31.224259 14376 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49186: Request ticket server kudu/127.4.82.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 01:00:31.224273 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 14185: Not authorized: Client connection negotiation failed: client connection to 127.4.82.131:45353: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.82.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 01:00:31.224462 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 14185
I20260809 01:00:31.244092 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 13812
I20260809 01:00:31.245433 13876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:31.366317 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 13812
2026-08-09T01:00:31Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21113 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T01:00:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:00:31Z Disabled control of system clock
I20260809 01:00:31.453012 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:33661
--webserver_interface=127.4.82.190
--webserver_port=0
--builtin_ntp_servers=127.4.82.148:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:33661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 01:00:31.781082 14381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:31.781335 14381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:31.781517 14381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:31.788477 14381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:00:31.788589 14381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:31.788694 14381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:31.788794 14381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:00:31.788857 14381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:00:31.797549 14381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:33661
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:33661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:31.798818 14381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:31.800557 14381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:31.812072 14389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:31.812129 14387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:31.812158 14386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:31.813074 14381 server_base.cc:1034] running on GCE node
I20260809 01:00:31.813796 14381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:31.815065 14381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:31.816294 14381 hybrid_clock.cc:648] HybridClock initialized: now 1786237231816259 us; error 59 us; skew 500 ppm
I20260809 01:00:31.816625 14381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:00:31.818729 14381 webserver.cc:460] Webserver started at http://127.4.82.190:40171/ using document root <none> and password file <none>
I20260809 01:00:31.819476 14381 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:31.819612 14381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:31.819936 14381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:31.822675 14381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "07af2d7575264cb490575bbe3dc782e9"
format_stamp: "Formatted at 2026-08-09 01:00:31 on dist-test-slave-07c2"
server_key: "b503171c1226d517ce186bc6edcfc2fc"
server_key_iv: "1755e1e97422802b6cfdbf890132f292"
server_key_version: "encryptionkey@0"
I20260809 01:00:31.823510 14381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "07af2d7575264cb490575bbe3dc782e9"
format_stamp: "Formatted at 2026-08-09 01:00:31 on dist-test-slave-07c2"
server_key: "b503171c1226d517ce186bc6edcfc2fc"
server_key_iv: "1755e1e97422802b6cfdbf890132f292"
server_key_version: "encryptionkey@0"
I20260809 01:00:31.828418 14381 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 01:00:31.831743 14395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:31.832958 14381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 01:00:31.833145 14381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "07af2d7575264cb490575bbe3dc782e9"
format_stamp: "Formatted at 2026-08-09 01:00:31 on dist-test-slave-07c2"
server_key: "b503171c1226d517ce186bc6edcfc2fc"
server_key_iv: "1755e1e97422802b6cfdbf890132f292"
server_key_version: "encryptionkey@0"
I20260809 01:00:31.833367 14381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:31.845623 14381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:31.846354 14381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:31.846633 14381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:31.865298 14381 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:33661
I20260809 01:00:31.865316 14448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:33661 every 8 connection(s)
I20260809 01:00:31.866619 14381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:00:31.870330 14449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:31.874383 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 14381
I20260809 01:00:31.874588 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:00:31.875268 4426 external_mini_cluster.cc:1428] Setting key 9f293d36380cff3de43241ecc7e5e8d6
I20260809 01:00:31.881078 14449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9: Bootstrap starting.
I20260809 01:00:31.884685 14449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:31.886014 14449 log.cc:824] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9: Log is configured to *not* fsync() on all Append() calls
I20260809 01:00:31.889098 14449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9: No bootstrap required, opened a new log
I20260809 01:00:31.889542 14452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:31.877227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58904 (local address 127.4.82.190:33661)
0809 01:00:31.878191 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:31.878201 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:31.878251 (+ 50us) server_negotiation.cc:407] Connection header received
0809 01:00:31.879941 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:31.879961 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:31.880374 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:31.880670 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:31.881378 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:31.883441 (+ 2063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:31.884733 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:31.885813 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:31.885996 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:31.887189 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:31.887203 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:31.887220 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:31.887820 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:31.887891 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:31.887911 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:31.888004 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:31.888262 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 01:00:31.888593 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":388,"threads_started":1}
I20260809 01:00:31.896375 14449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af2d7575264cb490575bbe3dc782e9" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33661 } }
I20260809 01:00:31.896714 14449 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:31.896816 14449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07af2d7575264cb490575bbe3dc782e9, State: Initialized, Role: FOLLOWER
I20260809 01:00:31.897408 14449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af2d7575264cb490575bbe3dc782e9" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33661 } }
I20260809 01:00:31.897644 14449 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:00:31.897789 14449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:00:31.897961 14449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:31.900182 14449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af2d7575264cb490575bbe3dc782e9" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33661 } }
I20260809 01:00:31.900650 14449 leader_election.cc:302] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07af2d7575264cb490575bbe3dc782e9; no voters:
I20260809 01:00:31.901206 14449 leader_election.cc:288] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:00:31.901345 14455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:00:31.902105 14455 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [term 1 LEADER]: Becoming Leader. State: Replica: 07af2d7575264cb490575bbe3dc782e9, State: Running, Role: LEADER
I20260809 01:00:31.902637 14455 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af2d7575264cb490575bbe3dc782e9" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33661 } }
I20260809 01:00:31.903290 14449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:00:31.905903 14456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07af2d7575264cb490575bbe3dc782e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af2d7575264cb490575bbe3dc782e9" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33661 } } }
I20260809 01:00:31.905874 14457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07af2d7575264cb490575bbe3dc782e9. Latest consensus state: current_term: 1 leader_uuid: "07af2d7575264cb490575bbe3dc782e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07af2d7575264cb490575bbe3dc782e9" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33661 } } }
I20260809 01:00:31.906344 14456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:31.906409 14457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:31.908007 14463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:00:31.913415 14463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:00:31.921797 14463 catalog_manager.cc:1347] Generated new cluster ID: 8d44ca91d72e4cc18c5f4e91ae3f14f5
I20260809 01:00:31.921908 14463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:00:31.935608 14463 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:00:31.937186 14463 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:00:31.944095 14463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07af2d7575264cb490575bbe3dc782e9: Generated new TSK 0
I20260809 01:00:31.944868 14463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:00:31.952351 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:33661
--builtin_ntp_servers=127.4.82.148:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 01:00:32.301331 14474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:32.301586 14474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:32.301669 14474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:32.308952 14474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:32.309041 14474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:32.309145 14474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:00:32.317734 14474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:33661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:32.318903 14474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:32.320554 14474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:32.332955 14480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:32.333046 14482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:32.333006 14479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:32.333739 14474 server_base.cc:1034] running on GCE node
I20260809 01:00:32.334316 14474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:32.335163 14474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:32.336385 14474 hybrid_clock.cc:648] HybridClock initialized: now 1786237232336338 us; error 58 us; skew 500 ppm
I20260809 01:00:32.336746 14474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:00:32.339097 14474 webserver.cc:460] Webserver started at http://127.4.82.129:46009/ using document root <none> and password file <none>
I20260809 01:00:32.339913 14474 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:32.340057 14474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:32.340380 14474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:32.343001 14474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "5af836955c8b4e07b58ad6b48b3f6444"
format_stamp: "Formatted at 2026-08-09 01:00:32 on dist-test-slave-07c2"
server_key: "4e95299c6308cda9bc396ff713b5d662"
server_key_iv: "ddc2526f4aa54939d7e4e8dc1715ded2"
server_key_version: "encryptionkey@0"
I20260809 01:00:32.343803 14474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "5af836955c8b4e07b58ad6b48b3f6444"
format_stamp: "Formatted at 2026-08-09 01:00:32 on dist-test-slave-07c2"
server_key: "4e95299c6308cda9bc396ff713b5d662"
server_key_iv: "ddc2526f4aa54939d7e4e8dc1715ded2"
server_key_version: "encryptionkey@0"
I20260809 01:00:32.348577 14474 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 01:00:32.351722 14488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:32.352905 14474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 01:00:32.353106 14474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "5af836955c8b4e07b58ad6b48b3f6444"
format_stamp: "Formatted at 2026-08-09 01:00:32 on dist-test-slave-07c2"
server_key: "4e95299c6308cda9bc396ff713b5d662"
server_key_iv: "ddc2526f4aa54939d7e4e8dc1715ded2"
server_key_version: "encryptionkey@0"
I20260809 01:00:32.353317 14474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:32.376549 14474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:32.377308 14474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:32.377607 14474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:32.378490 14474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:32.379860 14474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:32.379974 14474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:32.380100 14474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:32.380187 14474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:32.413149 14474 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:35553
I20260809 01:00:32.413242 14604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:35553 every 8 connection(s)
I20260809 01:00:32.414402 14474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:00:32.422760 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 14474
I20260809 01:00:32.422953 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:00:32.423651 4426 external_mini_cluster.cc:1428] Setting key 64bf03b64922e783961345dd399ffc48
I20260809 01:00:32.426465 14607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:32.416843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:35825 (local address 127.4.82.190:33661)
0809 01:00:32.417313 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:32.417317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:32.418295 (+ 978us) server_negotiation.cc:407] Connection header received
0809 01:00:32.420104 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:32.420110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:32.420197 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:32.420332 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:32.421280 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:32.421979 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:32.423668 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:32.424126 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:32.424285 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:32.425749 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:32.425753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:32.425756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:32.425773 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:32.425812 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:32.425824 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:32.425895 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:32.426141 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 01:00:32.426218 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":283,"threads_started":1}
I20260809 01:00:32.426674 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:33661
--builtin_ntp_servers=127.4.82.148:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 01:00:32.427826 14608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:32.417304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33661 (local address 127.4.82.129:35825)
0809 01:00:32.418144 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:32.418177 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:32.419847 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:32.420539 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:32.420546 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:32.420626 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:32.421117 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:32.421131 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:32.422149 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:32.422153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:32.423529 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:32.423541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:32.424350 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:32.424355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:32.424659 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:32.425416 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:00:32.425419 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:32.425468 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:00:32.425554 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:32.425928 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:32.425938 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:32.425946 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 01:00:32.426136 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 01:00:32.426511 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":262,"threads_started":1}
I20260809 01:00:32.429231 14605 heartbeater.cc:347] Connected to a master server at 127.4.82.190:33661
I20260809 01:00:32.429507 14605 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:32.430161 14605 heartbeater.cc:511] Master 127.4.82.190:33661 requested a full tablet report, sending...
I20260809 01:00:32.432094 14412 ts_manager.cc:194] Registered new tserver with Master: 5af836955c8b4e07b58ad6b48b3f6444 (127.4.82.129:35553)
I20260809 01:00:32.433547 14412 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.82.129:35825
W20260809 01:00:32.755119 14609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:32.755426 14609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:32.755592 14609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:32.762645 14609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:32.762769 14609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:32.762943 14609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:00:32.771835 14609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:33661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:32.773034 14609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:32.774770 14609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:32.787514 14615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:32.787565 14616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:32.787611 14618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:32.788298 14609 server_base.cc:1034] running on GCE node
I20260809 01:00:32.788925 14609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:32.789845 14609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:32.791069 14609 hybrid_clock.cc:648] HybridClock initialized: now 1786237232791026 us; error 54 us; skew 500 ppm
I20260809 01:00:32.791436 14609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:00:32.793618 14609 webserver.cc:460] Webserver started at http://127.4.82.130:33285/ using document root <none> and password file <none>
I20260809 01:00:32.794269 14609 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:32.794404 14609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:32.794732 14609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:32.797425 14609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "752f176d708e4f239a74c5a1c8c63506"
format_stamp: "Formatted at 2026-08-09 01:00:32 on dist-test-slave-07c2"
server_key: "68926b7c586b836bddaf752a58f5555e"
server_key_iv: "084d44ab9c34b42d311d6cfb494c9227"
server_key_version: "encryptionkey@0"
I20260809 01:00:32.798200 14609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "752f176d708e4f239a74c5a1c8c63506"
format_stamp: "Formatted at 2026-08-09 01:00:32 on dist-test-slave-07c2"
server_key: "68926b7c586b836bddaf752a58f5555e"
server_key_iv: "084d44ab9c34b42d311d6cfb494c9227"
server_key_version: "encryptionkey@0"
I20260809 01:00:32.803103 14609 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 01:00:32.806525 14624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:32.807809 14609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:00:32.807967 14609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "752f176d708e4f239a74c5a1c8c63506"
format_stamp: "Formatted at 2026-08-09 01:00:32 on dist-test-slave-07c2"
server_key: "68926b7c586b836bddaf752a58f5555e"
server_key_iv: "084d44ab9c34b42d311d6cfb494c9227"
server_key_version: "encryptionkey@0"
I20260809 01:00:32.808168 14609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:32.839031 14609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:32.839854 14609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:32.840143 14609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:32.841004 14609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:32.842435 14609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:32.842561 14609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:32.842692 14609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:32.842782 14609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:32.876348 14609 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:32841
I20260809 01:00:32.876410 14739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:32841 every 8 connection(s)
I20260809 01:00:32.877586 14609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:00:32.882427 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 14609
I20260809 01:00:32.882601 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:00:32.883087 4426 external_mini_cluster.cc:1428] Setting key 42b841567241a941f7855f0072df7f74
I20260809 01:00:32.885596 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:33661
--builtin_ntp_servers=127.4.82.148:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 01:00:32.894770 14607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:32.880145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:45619 (local address 127.4.82.190:33661)
0809 01:00:32.880344 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:32.880351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:32.881742 (+ 1391us) server_negotiation.cc:407] Connection header received
0809 01:00:32.884088 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:32.884094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:32.884181 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:32.884295 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:32.885310 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:32.886099 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:32.887667 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:32.888053 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:32.888192 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:32.893964 (+ 5772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:32.893970 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:32.893973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:32.893990 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:32.894021 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:32.894032 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:32.894135 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:32.894364 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 01:00:32.894463 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 01:00:32.895848 14742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:32.880604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33661 (local address 127.4.82.130:45619)
0809 01:00:32.881588 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:32.881630 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:32.883803 (+ 2173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:32.884516 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:32.884523 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:32.884610 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:32.885147 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:32.885163 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:32.886253 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:32.886260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:32.887522 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:32.887534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:32.888285 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:32.888293 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:32.888510 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:32.893655 (+ 5145us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:00:32.893658 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:32.893695 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:00:32.893766 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:32.894136 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:32.894149 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:32.894159 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 01:00:32.894335 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 01:00:32.894722 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":285,"threads_started":1}
I20260809 01:00:32.897243 14740 heartbeater.cc:347] Connected to a master server at 127.4.82.190:33661
I20260809 01:00:32.897539 14740 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:32.898237 14740 heartbeater.cc:511] Master 127.4.82.190:33661 requested a full tablet report, sending...
I20260809 01:00:32.899586 14412 ts_manager.cc:194] Registered new tserver with Master: 752f176d708e4f239a74c5a1c8c63506 (127.4.82.130:32841)
I20260809 01:00:32.900359 14412 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.82.130:45619
W20260809 01:00:33.221117 14743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:33.221397 14743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:33.221554 14743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:33.228567 14743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:33.228684 14743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:33.228856 14743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:00:33.237540 14743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:33661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:33.238703 14743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:33.240391 14743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:33.252884 14748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:33.252844 14749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:33.252795 14751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:33.254102 14743 server_base.cc:1034] running on GCE node
I20260809 01:00:33.254668 14743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:33.255612 14743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:33.256839 14743 hybrid_clock.cc:648] HybridClock initialized: now 1786237233256787 us; error 69 us; skew 500 ppm
I20260809 01:00:33.257184 14743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:00:33.259379 14743 webserver.cc:460] Webserver started at http://127.4.82.131:38469/ using document root <none> and password file <none>
I20260809 01:00:33.260005 14743 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:33.260095 14743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:33.260367 14743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:33.262991 14743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "b5974c8c526c4d9596dd5103b59c3367"
format_stamp: "Formatted at 2026-08-09 01:00:33 on dist-test-slave-07c2"
server_key: "a0a691dcc18c986c75cc10e610bae7d2"
server_key_iv: "9b31d9510532d1fb49c17daadb2caf84"
server_key_version: "encryptionkey@0"
I20260809 01:00:33.263792 14743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "b5974c8c526c4d9596dd5103b59c3367"
format_stamp: "Formatted at 2026-08-09 01:00:33 on dist-test-slave-07c2"
server_key: "a0a691dcc18c986c75cc10e610bae7d2"
server_key_iv: "9b31d9510532d1fb49c17daadb2caf84"
server_key_version: "encryptionkey@0"
I20260809 01:00:33.268606 14743 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 01:00:33.271757 14757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:33.272917 14743 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260809 01:00:33.273087 14743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "b5974c8c526c4d9596dd5103b59c3367"
format_stamp: "Formatted at 2026-08-09 01:00:33 on dist-test-slave-07c2"
server_key: "a0a691dcc18c986c75cc10e610bae7d2"
server_key_iv: "9b31d9510532d1fb49c17daadb2caf84"
server_key_version: "encryptionkey@0"
I20260809 01:00:33.273289 14743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:33.287361 14743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:33.288090 14743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:33.288389 14743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:33.289273 14743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:33.290589 14743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:33.290714 14743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:33.290846 14743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:33.290935 14743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:33.323761 14743 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:44639
I20260809 01:00:33.323820 14877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:44639 every 8 connection(s)
I20260809 01:00:33.325014 14743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:00:33.331483 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 14743
I20260809 01:00:33.331655 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:00:33.332108 4426 external_mini_cluster.cc:1428] Setting key 8a8cbbf6eba6b2465fe63acc3a90cdf8
I20260809 01:00:33.337513 14607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:33.327689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:43109 (local address 127.4.82.190:33661)
0809 01:00:33.327857 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:33.327861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:33.329179 (+ 1318us) server_negotiation.cc:407] Connection header received
0809 01:00:33.330896 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:33.330902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:33.330991 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:33.331098 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:33.332197 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:33.333086 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:33.334578 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:33.334971 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:33.335106 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:33.336476 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:33.336487 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:33.336491 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:33.336508 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:33.336559 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:33.336573 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:33.336674 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:33.336929 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 01:00:33.337072 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 01:00:33.338845 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:33.328169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33661 (local address 127.4.82.131:43109)
0809 01:00:33.329032 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:33.329066 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:33.330641 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:33.331397 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:33.331405 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:33.331486 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:33.332031 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:33.332047 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:33.333237 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:33.333241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:33.334438 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:33.334451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:33.335203 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:33.335210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:33.335488 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:33.336163 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:00:33.336166 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:33.336207 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:00:33.336279 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:33.336682 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:33.336695 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:33.336706 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 01:00:33.336880 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 01:00:33.337295 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":306,"threads_started":1}
I20260809 01:00:33.340279 14878 heartbeater.cc:347] Connected to a master server at 127.4.82.190:33661
I20260809 01:00:33.340571 14878 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:33.341365 14878 heartbeater.cc:511] Master 127.4.82.190:33661 requested a full tablet report, sending...
I20260809 01:00:33.342789 14412 ts_manager.cc:194] Registered new tserver with Master: b5974c8c526c4d9596dd5103b59c3367 (127.4.82.131:44639)
I20260809 01:00:33.343842 14412 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.82.131:43109
I20260809 01:00:33.347829 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 01:00:33.353991 14607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:33.352467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58914 (local address 127.4.82.190:33661)
0809 01:00:33.352618 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:33.352621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:33.353105 (+ 484us) server_negotiation.cc:407] Connection header received
0809 01:00:33.353245 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:33.353248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:33.353318 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:33.353393 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:33.353782 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58914: BlockingRecv error: recv got EOF from 127.0.0.1:58914 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 01:00:33.354694 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 14474
I20260809 01:00:33.362066 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:33.355707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55340 (local address 127.4.82.129:35553)
0809 01:00:33.356285 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:33.356291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:33.356313 (+ 22us) server_negotiation.cc:407] Connection header received
0809 01:00:33.356364 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:33.356369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:33.356530 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:33.356644 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:33.357014 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:33.358070 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:33.359233 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:33.359815 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:33.359915 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:33.360797 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:33.360804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:33.360814 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:33.361331 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:33.361371 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:33.361388 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:33.361471 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:33.361601 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 01:00:33.361730 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":282,"threads_started":1}
I20260809 01:00:33.363158 14600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:33.443727 14605 heartbeater.cc:503] Master 127.4.82.190:33661 was elected leader, sending a full tablet report...
I20260809 01:00:33.445361 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 14474
I20260809 01:00:33.462558 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 14609
I20260809 01:00:33.470472 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:33.463519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44254 (local address 127.4.82.130:32841)
0809 01:00:33.463993 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:33.463999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:33.464022 (+ 23us) server_negotiation.cc:407] Connection header received
0809 01:00:33.464080 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:33.464086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:33.464304 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:33.464470 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:33.464877 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:33.465902 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:33.467245 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:33.467930 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:33.468060 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:33.469084 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:33.469092 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:33.469104 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:33.469656 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:33.469700 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:33.469730 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:33.469866 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:33.470007 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 01:00:33.470139 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":224,"threads_started":1}
I20260809 01:00:33.471714 14735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:33.554814 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 14609
I20260809 01:00:33.574247 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 14743
I20260809 01:00:33.586979 14894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:33.575646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55888 (local address 127.4.82.131:44639)
0809 01:00:33.576796 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:33.576863 (+ 67us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:33.576949 (+ 86us) server_negotiation.cc:407] Connection header received
0809 01:00:33.577165 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:33.577175 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:33.577614 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:33.577795 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:33.578431 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:33.580302 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:33.582438 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:33.583443 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:33.583587 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:33.584790 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:33.584798 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:33.584812 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:33.585549 (+ 737us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:33.585626 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:33.585656 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:33.585769 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:33.586312 (+ 543us) server_negotiation.cc:299] Negotiation successful
0809 01:00:33.586503 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":859,"thread_start_us":552,"threads_started":1}
I20260809 01:00:33.588527 14873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:33.674141 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 14743
I20260809 01:00:33.693061 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 14381
I20260809 01:00:33.694427 14444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:33.796380 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 14381
2026-08-09T01:00:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2423 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T01:00:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:00:33Z Disabled control of system clock
I20260809 01:00:33.874374 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:42427
--webserver_interface=127.4.82.190
--webserver_port=0
--builtin_ntp_servers=127.4.82.148:36455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:42427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 01:00:34.197880 14900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:34.198139 14900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:34.198239 14900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:34.205389 14900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:00:34.205505 14900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:34.205600 14900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:34.205698 14900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:00:34.205778 14900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:00:34.215010 14900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:42427
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:42427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:34.216248 14900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:34.217962 14900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:34.229336 14905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:34.229302 14908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:34.229269 14906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:34.230131 14900 server_base.cc:1034] running on GCE node
I20260809 01:00:34.230865 14900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:34.232197 14900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:34.233402 14900 hybrid_clock.cc:648] HybridClock initialized: now 1786237234233361 us; error 28 us; skew 500 ppm
I20260809 01:00:34.233723 14900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:00:34.235827 14900 webserver.cc:460] Webserver started at http://127.4.82.190:42731/ using document root <none> and password file <none>
I20260809 01:00:34.236476 14900 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:34.236588 14900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:34.236909 14900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:34.239672 14900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "210a68cdac6e41279f7a2a4084ed12c3"
format_stamp: "Formatted at 2026-08-09 01:00:34 on dist-test-slave-07c2"
server_key: "74b9166b590104f08412b7735c2224de"
server_key_iv: "bec77fa85736fccd82b9b61d766484e0"
server_key_version: "encryptionkey@0"
I20260809 01:00:34.240442 14900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "210a68cdac6e41279f7a2a4084ed12c3"
format_stamp: "Formatted at 2026-08-09 01:00:34 on dist-test-slave-07c2"
server_key: "74b9166b590104f08412b7735c2224de"
server_key_iv: "bec77fa85736fccd82b9b61d766484e0"
server_key_version: "encryptionkey@0"
I20260809 01:00:34.245159 14900 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 01:00:34.248282 14914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:34.249421 14900 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 01:00:34.249593 14900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "210a68cdac6e41279f7a2a4084ed12c3"
format_stamp: "Formatted at 2026-08-09 01:00:34 on dist-test-slave-07c2"
server_key: "74b9166b590104f08412b7735c2224de"
server_key_iv: "bec77fa85736fccd82b9b61d766484e0"
server_key_version: "encryptionkey@0"
I20260809 01:00:34.249794 14900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:34.255106 14900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:34.255856 14900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:34.256134 14900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:34.273689 14900 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:42427
I20260809 01:00:34.273725 14974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:42427 every 8 connection(s)
I20260809 01:00:34.274909 14900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:00:34.278360 14975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:34.284783 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 14900
I20260809 01:00:34.285010 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:00:34.285840 4426 external_mini_cluster.cc:1428] Setting key 5e933c41732b2edaae389d5976080ef4
I20260809 01:00:34.289122 14975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3: Bootstrap starting.
I20260809 01:00:34.292418 14975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:34.293555 14975 log.cc:824] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3: Log is configured to *not* fsync() on all Append() calls
I20260809 01:00:34.293848 14978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:34.287833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45362 (local address 127.4.82.190:42427)
0809 01:00:34.288784 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:34.288794 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:34.288833 (+ 39us) server_negotiation.cc:407] Connection header received
0809 01:00:34.290495 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:34.290516 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:34.290895 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:34.291181 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:34.292300 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:34.292307 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:34.292318 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:34.292335 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:34.292390 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:34.292408 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:34.292611 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:34.292723 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 01:00:34.292989 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":331,"threads_started":1}
I20260809 01:00:34.297025 14975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3: No bootstrap required, opened a new log
I20260809 01:00:34.303946 14975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210a68cdac6e41279f7a2a4084ed12c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 42427 } }
I20260809 01:00:34.304268 14975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:34.304381 14975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210a68cdac6e41279f7a2a4084ed12c3, State: Initialized, Role: FOLLOWER
I20260809 01:00:34.304945 14975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210a68cdac6e41279f7a2a4084ed12c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 42427 } }
I20260809 01:00:34.305172 14975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:00:34.305320 14975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:00:34.305491 14975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:34.307821 14975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210a68cdac6e41279f7a2a4084ed12c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 42427 } }
I20260809 01:00:34.308288 14975 leader_election.cc:302] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 210a68cdac6e41279f7a2a4084ed12c3; no voters:
I20260809 01:00:34.308845 14975 leader_election.cc:288] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:00:34.308995 14980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:00:34.309744 14980 raft_consensus.cc:686] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [term 1 LEADER]: Becoming Leader. State: Replica: 210a68cdac6e41279f7a2a4084ed12c3, State: Running, Role: LEADER
I20260809 01:00:34.310298 14980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210a68cdac6e41279f7a2a4084ed12c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 42427 } }
I20260809 01:00:34.310827 14975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:00:34.313143 14981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "210a68cdac6e41279f7a2a4084ed12c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210a68cdac6e41279f7a2a4084ed12c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 42427 } } }
I20260809 01:00:34.313481 14981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:34.313213 14982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 210a68cdac6e41279f7a2a4084ed12c3. Latest consensus state: current_term: 1 leader_uuid: "210a68cdac6e41279f7a2a4084ed12c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210a68cdac6e41279f7a2a4084ed12c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 42427 } } }
I20260809 01:00:34.314459 14982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:34.315778 14987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:00:34.320503 14987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:00:34.329887 14987 catalog_manager.cc:1347] Generated new cluster ID: 575b5de0e1b748dab47d1ea6c7334aa0
I20260809 01:00:34.329988 14987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:00:34.340415 14987 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:00:34.340794 14987 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:00:34.355348 14987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 210a68cdac6e41279f7a2a4084ed12c3: Generated new TSK 0
I20260809 01:00:34.356035 14987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:00:34.363958 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:42427
--builtin_ntp_servers=127.4.82.148:36455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 01:00:34.696895 15000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:34.697142 15000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:34.697247 15000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:34.704077 15000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:34.704161 15000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:34.704263 15000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:00:34.713001 15000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:42427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:34.714188 15000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:34.715912 15000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:34.728147 15006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:34.728147 15008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:34.728319 15005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:34.728826 15000 server_base.cc:1034] running on GCE node
I20260809 01:00:34.729454 15000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:34.730314 15000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:34.731500 15000 hybrid_clock.cc:648] HybridClock initialized: now 1786237234731466 us; error 36 us; skew 500 ppm
I20260809 01:00:34.731799 15000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:00:34.733976 15000 webserver.cc:460] Webserver started at http://127.4.82.129:45547/ using document root <none> and password file <none>
I20260809 01:00:34.734604 15000 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:34.734720 15000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:34.735028 15000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:34.739894 15000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "5c851a68d7f141438bf04884766da93f"
format_stamp: "Formatted at 2026-08-09 01:00:34 on dist-test-slave-07c2"
server_key: "7317487400708121563129e69cf27e60"
server_key_iv: "af9c88a088ee5032129c2540879e2031"
server_key_version: "encryptionkey@0"
I20260809 01:00:34.740724 15000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "5c851a68d7f141438bf04884766da93f"
format_stamp: "Formatted at 2026-08-09 01:00:34 on dist-test-slave-07c2"
server_key: "7317487400708121563129e69cf27e60"
server_key_iv: "af9c88a088ee5032129c2540879e2031"
server_key_version: "encryptionkey@0"
I20260809 01:00:34.745601 15000 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 01:00:34.748853 15014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:34.750166 15000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 01:00:34.750331 15000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "5c851a68d7f141438bf04884766da93f"
format_stamp: "Formatted at 2026-08-09 01:00:34 on dist-test-slave-07c2"
server_key: "7317487400708121563129e69cf27e60"
server_key_iv: "af9c88a088ee5032129c2540879e2031"
server_key_version: "encryptionkey@0"
I20260809 01:00:34.750537 15000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:34.756110 15000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:34.756858 15000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:34.757129 15000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:34.758005 15000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:34.759341 15000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:34.759464 15000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:34.759586 15000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:34.759687 15000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:34.790520 15000 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:37593
I20260809 01:00:34.790575 15134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:37593 every 8 connection(s)
I20260809 01:00:34.791806 15000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:00:34.794992 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 15000
I20260809 01:00:34.795219 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:00:34.796150 4426 external_mini_cluster.cc:1428] Setting key 593d625e2a5aab0b7c1b03ccb6d8544a
I20260809 01:00:34.798988 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:42427
--builtin_ntp_servers=127.4.82.148:36455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 01:00:34.800979 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:34.794644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:50141 (local address 127.4.82.190:42427)
0809 01:00:34.795122 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:34.795128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:34.796533 (+ 1405us) server_negotiation.cc:407] Connection header received
0809 01:00:34.798665 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:34.798673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:34.798768 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:34.798877 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:34.800237 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:34.800243 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:34.800248 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:34.800268 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:34.800309 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:34.800321 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:34.800444 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:34.800634 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 01:00:34.800699 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":257,"threads_started":1}
I20260809 01:00:34.801851 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:34.795390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:42427 (local address 127.4.82.129:50141)
0809 01:00:34.796389 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:34.796424 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:34.798412 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:34.799114 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:34.799128 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:34.799244 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:34.799969 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:00:34.799971 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:34.800004 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:00:34.800080 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:34.800413 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:34.800425 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:34.800432 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 01:00:34.800712 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 01:00:34.800937 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":332,"threads_started":1}
I20260809 01:00:34.803305 15135 heartbeater.cc:347] Connected to a master server at 127.4.82.190:42427
I20260809 01:00:34.803611 15135 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:34.804291 15135 heartbeater.cc:511] Master 127.4.82.190:42427 requested a full tablet report, sending...
I20260809 01:00:34.806509 14937 ts_manager.cc:194] Registered new tserver with Master: 5c851a68d7f141438bf04884766da93f (127.4.82.129:37593)
W20260809 01:00:35.121188 15139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:35.121465 15139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:35.121640 15139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:35.129532 15139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:35.129640 15139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:35.129824 15139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:00:35.138996 15139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:42427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:35.140334 15139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:35.142117 15139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:35.155661 15144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:35.155718 15145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:35.155750 15147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:35.157039 15139 server_base.cc:1034] running on GCE node
I20260809 01:00:35.157603 15139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:35.158445 15139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:35.159669 15139 hybrid_clock.cc:648] HybridClock initialized: now 1786237235159613 us; error 59 us; skew 500 ppm
I20260809 01:00:35.159991 15139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:00:35.162070 15139 webserver.cc:460] Webserver started at http://127.4.82.130:34033/ using document root <none> and password file <none>
I20260809 01:00:35.162726 15139 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:35.162849 15139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:35.163167 15139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:35.165922 15139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "786e6a3bbbd241c8be51b99c8590bb24"
format_stamp: "Formatted at 2026-08-09 01:00:35 on dist-test-slave-07c2"
server_key: "e9d42135f998dd1b0e9601b1929e63ef"
server_key_iv: "7231ffa0b365f9851c3016e15e636b19"
server_key_version: "encryptionkey@0"
I20260809 01:00:35.166674 15139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "786e6a3bbbd241c8be51b99c8590bb24"
format_stamp: "Formatted at 2026-08-09 01:00:35 on dist-test-slave-07c2"
server_key: "e9d42135f998dd1b0e9601b1929e63ef"
server_key_iv: "7231ffa0b365f9851c3016e15e636b19"
server_key_version: "encryptionkey@0"
I20260809 01:00:35.171615 15139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 01:00:35.174757 15153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:35.175976 15139 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 01:00:35.176141 15139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "786e6a3bbbd241c8be51b99c8590bb24"
format_stamp: "Formatted at 2026-08-09 01:00:35 on dist-test-slave-07c2"
server_key: "e9d42135f998dd1b0e9601b1929e63ef"
server_key_iv: "7231ffa0b365f9851c3016e15e636b19"
server_key_version: "encryptionkey@0"
I20260809 01:00:35.176352 15139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:35.181638 15139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:35.182344 15139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:35.182601 15139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:35.183488 15139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:35.184824 15139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:35.184943 15139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:35.185058 15139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:35.185141 15139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:35.217183 15139 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:45307
I20260809 01:00:35.217259 15271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:45307 every 8 connection(s)
I20260809 01:00:35.218407 15139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:00:35.225193 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 15139
I20260809 01:00:35.225437 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:00:35.225950 4426 external_mini_cluster.cc:1428] Setting key c3fe0b1fd3b2f73124bc2b9bb8b449c5
I20260809 01:00:35.226287 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:35.220813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:55593 (local address 127.4.82.190:42427)
0809 01:00:35.221035 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:35.221042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:35.222305 (+ 1263us) server_negotiation.cc:407] Connection header received
0809 01:00:35.223943 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:35.223947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:35.224000 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:35.224112 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:35.225539 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:35.225543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:35.225546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:35.225561 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:35.225591 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:35.225602 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:35.225737 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:35.225923 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 01:00:35.225992 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 01:00:35.227231 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:35.221306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:42427 (local address 127.4.82.130:55593)
0809 01:00:35.222171 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:35.222205 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:35.223723 (+ 1518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:35.224309 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:35.224332 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:35.224468 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:35.225205 (+ 737us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:00:35.225208 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:35.225245 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:00:35.225346 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:35.225692 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:35.225704 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:35.225721 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 01:00:35.226001 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 01:00:35.226229 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":280,"threads_started":1}
I20260809 01:00:35.228662 15272 heartbeater.cc:347] Connected to a master server at 127.4.82.190:42427
I20260809 01:00:35.228926 15272 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:35.229149 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:42427
--builtin_ntp_servers=127.4.82.148:36455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 01:00:35.229506 15272 heartbeater.cc:511] Master 127.4.82.190:42427 requested a full tablet report, sending...
I20260809 01:00:35.230772 14937 ts_manager.cc:194] Registered new tserver with Master: 786e6a3bbbd241c8be51b99c8590bb24 (127.4.82.130:45307)
W20260809 01:00:35.557247 15275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:35.557504 15275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:35.557660 15275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:35.564853 15275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:35.564940 15275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:35.565127 15275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:00:35.575927 15275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:42427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:35.577633 15275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:35.579772 15275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:35.592959 15280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:35.592990 15283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:35.593061 15281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:35.595064 15275 server_base.cc:1034] running on GCE node
I20260809 01:00:35.595779 15275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:35.597386 15275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:35.598662 15275 hybrid_clock.cc:648] HybridClock initialized: now 1786237235598595 us; error 47 us; skew 500 ppm
I20260809 01:00:35.599115 15275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:00:35.601431 15275 webserver.cc:460] Webserver started at http://127.4.82.131:35781/ using document root <none> and password file <none>
I20260809 01:00:35.602101 15275 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:35.602193 15275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:35.602478 15275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:35.605460 15275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "65509018c45d4ee0945aeae3b89b59ea"
format_stamp: "Formatted at 2026-08-09 01:00:35 on dist-test-slave-07c2"
server_key: "3531f827b839e6be9f6e3c423b91741e"
server_key_iv: "6b41723399c8c1eea8f2ba3c71cbaaf9"
server_key_version: "encryptionkey@0"
I20260809 01:00:35.606226 15275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "65509018c45d4ee0945aeae3b89b59ea"
format_stamp: "Formatted at 2026-08-09 01:00:35 on dist-test-slave-07c2"
server_key: "3531f827b839e6be9f6e3c423b91741e"
server_key_iv: "6b41723399c8c1eea8f2ba3c71cbaaf9"
server_key_version: "encryptionkey@0"
I20260809 01:00:35.611620 15275 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 01:00:35.615088 15289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:35.616526 15275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:00:35.616732 15275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "65509018c45d4ee0945aeae3b89b59ea"
format_stamp: "Formatted at 2026-08-09 01:00:35 on dist-test-slave-07c2"
server_key: "3531f827b839e6be9f6e3c423b91741e"
server_key_iv: "6b41723399c8c1eea8f2ba3c71cbaaf9"
server_key_version: "encryptionkey@0"
I20260809 01:00:35.616962 15275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:35.622772 15275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:35.623641 15275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:35.623911 15275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:35.624917 15275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:35.626360 15275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:35.626487 15275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:35.626613 15275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:35.626705 15275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:35.660748 15275 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:44359
I20260809 01:00:35.660800 15405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:44359 every 8 connection(s)
I20260809 01:00:35.662122 15275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:00:35.665004 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 15275
I20260809 01:00:35.665228 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:00:35.665668 4426 external_mini_cluster.cc:1428] Setting key 1f1bd20d9213cc94b544166811bb5e34
I20260809 01:00:35.671057 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:35.664912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:45587 (local address 127.4.82.190:42427)
0809 01:00:35.665176 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:35.665185 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:35.666812 (+ 1627us) server_negotiation.cc:407] Connection header received
0809 01:00:35.669007 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:35.669011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:35.669063 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:35.669144 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:35.670424 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:35.670429 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:35.670432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:35.670446 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:35.670476 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:35.670488 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:35.670577 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:35.670752 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 01:00:35.670802 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 01:00:35.672114 15408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:35.665491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:42427 (local address 127.4.82.131:45587)
0809 01:00:35.666632 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:35.666694 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:35.668717 (+ 2023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:35.669360 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:35.669368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:35.669456 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:35.670157 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:00:35.670165 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:35.670202 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:00:35.670282 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:35.670570 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:35.670577 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:35.670582 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 01:00:35.670802 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 01:00:35.671031 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":399,"threads_started":1}
I20260809 01:00:35.673381 15406 heartbeater.cc:347] Connected to a master server at 127.4.82.190:42427
I20260809 01:00:35.673657 15406 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:35.674266 15406 heartbeater.cc:511] Master 127.4.82.190:42427 requested a full tablet report, sending...
I20260809 01:00:35.675652 14937 ts_manager.cc:194] Registered new tserver with Master: 65509018c45d4ee0945aeae3b89b59ea (127.4.82.131:44359)
I20260809 01:00:35.681066 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 01:00:35.687137 15137 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:00:35.685571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45376 (local address 127.4.82.190:42427)
0809 01:00:35.685733 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:35.685737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:35.686241 (+ 504us) server_negotiation.cc:407] Connection header received
0809 01:00:35.686424 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:35.686428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:35.686479 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:35.686575 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:35.686940 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45376: BlockingRecv error: recv got EOF from 127.0.0.1:45376 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 01:00:35.688201 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 15000
I20260809 01:00:35.691913 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:35.689206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.4.82.129:37593)
0809 01:00:35.689738 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:35.689745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:35.689768 (+ 23us) server_negotiation.cc:407] Connection header received
0809 01:00:35.689829 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:35.689834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:35.690025 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:35.690178 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:35.691172 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:35.691179 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:35.691189 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:35.691207 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:35.691253 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:35.691267 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:35.691434 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:35.691535 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 01:00:35.691637 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":246,"threads_started":1}
I20260809 01:00:35.693104 15130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:35.773213 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 15000
I20260809 01:00:35.790416 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 15139
I20260809 01:00:35.793808 15417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:35.791274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41200 (local address 127.4.82.130:45307)
0809 01:00:35.791888 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:35.791895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:35.791923 (+ 28us) server_negotiation.cc:407] Connection header received
0809 01:00:35.791985 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:35.791991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:35.792207 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:35.792337 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:35.793210 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:35.793216 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:35.793226 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:35.793244 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:35.793282 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:35.793296 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:35.793396 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:35.793474 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 01:00:35.793553 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":286,"threads_started":1}
I20260809 01:00:35.795009 15267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:35.875154 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 15139
I20260809 01:00:35.892436 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 15275
I20260809 01:00:35.896196 15419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:35.893459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43122 (local address 127.4.82.131:44359)
0809 01:00:35.894074 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:35.894081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:35.894110 (+ 29us) server_negotiation.cc:407] Connection header received
0809 01:00:35.894171 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:35.894177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:35.894395 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:35.894518 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:35.895447 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:35.895454 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:35.895465 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:00:35.895478 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:35.895520 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:00:35.895533 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:35.895664 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:35.895746 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 01:00:35.895862 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":317,"threads_started":1}
I20260809 01:00:35.897388 15401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:35.977932 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 15275
I20260809 01:00:35.996183 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 14900
I20260809 01:00:35.997377 14970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:36.098238 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 14900
2026-08-09T01:00:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2302 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:00:36 dist-test-slave-07c2 krb5kdc[15424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:00:36 dist-test-slave-07c2 krb5kdc[15424](info): set up 2 sockets
Aug 09 01:00:36 dist-test-slave-07c2 krb5kdc[15424](info): commencing operation
krb5kdc: starting...
W20260809 01:00:37.228282 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:00:37 dist-test-slave-07c2 krb5kdc[15424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:00:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:00:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:00:37.478497 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:37893
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:37893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:37.806700 15437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:37.806985 15437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:37.807170 15437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:37.814334 15437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:00:37.814432 15437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:37.814479 15437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:37.814563 15437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:00:37.814635 15437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:00:37.824329 15437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:37893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:37893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:37.825626 15437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:37.826926 15437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:37.840068 15444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:37.840358 15443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:37.840443 15446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:37.842301 15437 server_base.cc:1034] running on GCE node
I20260809 01:00:37.843290 15437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:37.844771 15437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:37.845966 15437 hybrid_clock.cc:648] HybridClock initialized: now 1786237237845921 us; error 41 us; skew 500 ppm
I20260809 01:00:37.846300 15437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:37 dist-test-slave-07c2 krb5kdc[15424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237237, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:37.849414 15437 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:00:37.850816 15437 webserver.cc:460] Webserver started at http://127.4.82.190:44155/ using document root <none> and password file <none>
I20260809 01:00:37.851537 15437 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:37.851677 15437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:37.851979 15437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:37.854892 15437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "edd4c13d465546b2a8947eb7282a8605"
format_stamp: "Formatted at 2026-08-09 01:00:37 on dist-test-slave-07c2"
server_key: "6bf816174c5ab328a156ff1a70b438fd"
server_key_iv: "3d9526a55cb5f75143c3549ed00b3c18"
server_key_version: "encryptionkey@0"
I20260809 01:00:37.855691 15437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "edd4c13d465546b2a8947eb7282a8605"
format_stamp: "Formatted at 2026-08-09 01:00:37 on dist-test-slave-07c2"
server_key: "6bf816174c5ab328a156ff1a70b438fd"
server_key_iv: "3d9526a55cb5f75143c3549ed00b3c18"
server_key_version: "encryptionkey@0"
I20260809 01:00:37.860796 15437 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 01:00:37.864122 15453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:37.865289 15437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 01:00:37.865469 15437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "edd4c13d465546b2a8947eb7282a8605"
format_stamp: "Formatted at 2026-08-09 01:00:37 on dist-test-slave-07c2"
server_key: "6bf816174c5ab328a156ff1a70b438fd"
server_key_iv: "3d9526a55cb5f75143c3549ed00b3c18"
server_key_version: "encryptionkey@0"
I20260809 01:00:37.865669 15437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:37.891532 15437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:37.898443 15437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:37.898751 15437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:37.917145 15437 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:37893
I20260809 01:00:37.917181 15509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:37893 every 8 connection(s)
I20260809 01:00:37.918480 15437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:00:37.920058 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 15437
I20260809 01:00:37.920300 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:00:37.920934 4426 external_mini_cluster.cc:1428] Setting key 41d23c3d667099028b7cd5305a9e12d7
I20260809 01:00:37.922487 15511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 01:00:37 dist-test-slave-07c2 krb5kdc[15424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:00:37.936264 15511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605: Bootstrap starting.
I20260809 01:00:37.939714 15511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:37.940920 15511 log.cc:824] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605: Log is configured to *not* fsync() on all Append() calls
I20260809 01:00:37.944080 15511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605: No bootstrap required, opened a new log
I20260809 01:00:37.944530 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:37.922898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52878 (local address 127.4.82.190:37893)
0809 01:00:37.923753 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:37.923781 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:37.923833 (+ 52us) server_negotiation.cc:407] Connection header received
0809 01:00:37.925863 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:37.925890 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:37.926396 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:37.926815 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:37.928277 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:37.930338 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:37.931597 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:37.932627 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:37.932813 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:37.937084 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:37.937091 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:37.937104 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:37.937142 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:37.940208 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:37.940983 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:37.940989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:37.940994 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:37.941189 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:37.941668 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:37.941673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:37.941678 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:37.942407 (+ 729us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:37.942625 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:37.943091 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 01:00:37.943515 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":417,"threads_started":1}
I20260809 01:00:37.951198 15511 raft_consensus.cc:348] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edd4c13d465546b2a8947eb7282a8605" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37893 } }
I20260809 01:00:37.951594 15511 raft_consensus.cc:374] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:37.951704 15511 raft_consensus.cc:729] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edd4c13d465546b2a8947eb7282a8605, State: Initialized, Role: FOLLOWER
I20260809 01:00:37.952224 15511 consensus_queue.cc:260] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edd4c13d465546b2a8947eb7282a8605" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37893 } }
I20260809 01:00:37.952452 15511 raft_consensus.cc:388] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:00:37.952591 15511 raft_consensus.cc:482] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:00:37.952764 15511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:37.954880 15511 raft_consensus.cc:504] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edd4c13d465546b2a8947eb7282a8605" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37893 } }
I20260809 01:00:37.955420 15511 leader_election.cc:302] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edd4c13d465546b2a8947eb7282a8605; no voters:
I20260809 01:00:37.955991 15511 leader_election.cc:288] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:00:37.956128 15516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:00:37.956807 15516 raft_consensus.cc:686] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [term 1 LEADER]: Becoming Leader. State: Replica: edd4c13d465546b2a8947eb7282a8605, State: Running, Role: LEADER
I20260809 01:00:37.957352 15516 consensus_queue.cc:237] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edd4c13d465546b2a8947eb7282a8605" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37893 } }
I20260809 01:00:37.958112 15511 sys_catalog.cc:564] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:00:37.959885 15518 sys_catalog.cc:455] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edd4c13d465546b2a8947eb7282a8605. Latest consensus state: current_term: 1 leader_uuid: "edd4c13d465546b2a8947eb7282a8605" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edd4c13d465546b2a8947eb7282a8605" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37893 } } }
I20260809 01:00:37.960237 15518 sys_catalog.cc:458] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:37.960989 15517 sys_catalog.cc:455] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edd4c13d465546b2a8947eb7282a8605" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edd4c13d465546b2a8947eb7282a8605" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37893 } } }
I20260809 01:00:37.961234 15522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:00:37.961325 15517 sys_catalog.cc:458] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605 [sys.catalog]: This master's current role is: LEADER
I20260809 01:00:37.967304 15522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:00:37.975816 15522 catalog_manager.cc:1347] Generated new cluster ID: 7fce952a76154802a2b59778553799d8
I20260809 01:00:37.976065 15522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:00:37.992872 15522 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:00:37.994680 15522 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:00:38.006192 15522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605: Generated new TSK 0
I20260809 01:00:38.007263 15522 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 01:00:38.074708 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37893
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:38.404708 15537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:38.404959 15537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:38.405144 15537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:38.412154 15537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:38.412269 15537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:38.412442 15537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:00:38.421628 15537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:38.422780 15537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:38.424043 15537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:38.436414 15545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:38.436465 15543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:38.436589 15542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:38.437471 15537 server_base.cc:1034] running on GCE node
I20260809 01:00:38.438083 15537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:38.438910 15537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:38.440119 15537 hybrid_clock.cc:648] HybridClock initialized: now 1786237238440044 us; error 52 us; skew 500 ppm
I20260809 01:00:38.440443 15537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:38 dist-test-slave-07c2 krb5kdc[15424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237238, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:38.443447 15537 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:00:38.444895 15537 webserver.cc:460] Webserver started at http://127.4.82.129:38349/ using document root <none> and password file <none>
I20260809 01:00:38.445528 15537 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:38.445672 15537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:38.445960 15537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:38.448761 15537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "9c63f0389c624470b3bf105ff7e13531"
format_stamp: "Formatted at 2026-08-09 01:00:38 on dist-test-slave-07c2"
server_key: "8fd79f85c1dffb7b6f3c67b12cd62d18"
server_key_iv: "ac2c0eaab86521d9a10aa3d2dca920de"
server_key_version: "encryptionkey@0"
I20260809 01:00:38.449532 15537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "9c63f0389c624470b3bf105ff7e13531"
format_stamp: "Formatted at 2026-08-09 01:00:38 on dist-test-slave-07c2"
server_key: "8fd79f85c1dffb7b6f3c67b12cd62d18"
server_key_iv: "ac2c0eaab86521d9a10aa3d2dca920de"
server_key_version: "encryptionkey@0"
I20260809 01:00:38.454281 15537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260809 01:00:38.457453 15554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:38.458896 15537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 01:00:38.459051 15537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "9c63f0389c624470b3bf105ff7e13531"
format_stamp: "Formatted at 2026-08-09 01:00:38 on dist-test-slave-07c2"
server_key: "8fd79f85c1dffb7b6f3c67b12cd62d18"
server_key_iv: "ac2c0eaab86521d9a10aa3d2dca920de"
server_key_version: "encryptionkey@0"
I20260809 01:00:38.459251 15537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:38.485416 15537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:38.489099 15537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:38.489374 15537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:38.490232 15537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:38.491519 15537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:38.491613 15537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:38.491755 15537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:38.491825 15537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:38.523507 15537 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:46559
I20260809 01:00:38.523545 15673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:46559 every 8 connection(s)
I20260809 01:00:38.524796 15537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:00:38.531737 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 15537
I20260809 01:00:38.531966 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:00:38.532603 4426 external_mini_cluster.cc:1428] Setting key a5fdb5afebf5d15145164d9b06fc0732
Aug 09 01:00:38 dist-test-slave-07c2 krb5kdc[15424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237238, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:00:38.546990 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:38.527318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:41869 (local address 127.4.82.190:37893)
0809 01:00:38.527783 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:38.527787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:38.528681 (+ 894us) server_negotiation.cc:407] Connection header received
0809 01:00:38.530495 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:38.530500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:38.530571 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:38.530666 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:38.532320 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:38.533092 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:38.534960 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:38.535417 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:38.535564 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:38.539366 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:38.539375 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:38.539382 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:38.539431 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:38.543141 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:38.544094 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:38.544102 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:38.544108 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:38.544271 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:38.544768 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:38.544775 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:38.544781 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:38.545180 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:38.545346 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:38.546254 (+ 908us) server_negotiation.cc:299] Negotiation successful
0809 01:00:38.546536 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":249,"threads_started":1}
I20260809 01:00:38.547711 15677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:38.527842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:37893 (local address 127.4.82.129:41869)
0809 01:00:38.528517 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:38.528564 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:38.530255 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:38.530865 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:38.530872 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:38.531498 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:38.532147 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:38.532172 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:38.533257 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:38.533262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:38.534802 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:38.534820 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:38.535656 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:38.535664 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:38.535918 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:38.536648 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:38.536673 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:38.539098 (+ 2425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:38.543330 (+ 4232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:38.543342 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:38.543361 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:38.543964 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:38.544390 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:38.544397 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:38.544402 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:38.544638 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:38.545305 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:38.545315 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:38.545761 (+ 446us) client_negotiation.cc:769] Sending connection context
0809 01:00:38.546083 (+ 322us) client_negotiation.cc:240] Negotiation successful
0809 01:00:38.546530 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":270,"threads_started":1}
I20260809 01:00:38.549330 15674 heartbeater.cc:347] Connected to a master server at 127.4.82.190:37893
I20260809 01:00:38.549697 15674 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:38.550603 15674 heartbeater.cc:511] Master 127.4.82.190:37893 requested a full tablet report, sending...
I20260809 01:00:38.552860 15472 ts_manager.cc:194] Registered new tserver with Master: 9c63f0389c624470b3bf105ff7e13531 (127.4.82.129:46559)
I20260809 01:00:38.554302 15472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:41869
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 01:00:38.595752 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37893
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:38.919953 15680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:38.920198 15680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:38.920346 15680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:38.927209 15680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:38.927347 15680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:38.927516 15680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:00:38.936704 15680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:38.937798 15680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:38.939009 15680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:38.951874 15685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:38.951824 15688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:38.951908 15686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:38.952662 15680 server_base.cc:1034] running on GCE node
I20260809 01:00:38.953176 15680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:38.954020 15680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:38.955247 15680 hybrid_clock.cc:648] HybridClock initialized: now 1786237238955204 us; error 48 us; skew 500 ppm
I20260809 01:00:38.955590 15680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:38 dist-test-slave-07c2 krb5kdc[15424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237238, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:38.958595 15680 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:00:38.960021 15680 webserver.cc:460] Webserver started at http://127.4.82.130:35999/ using document root <none> and password file <none>
I20260809 01:00:38.960659 15680 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:38.960799 15680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:38.961081 15680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:38.964145 15680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444"
format_stamp: "Formatted at 2026-08-09 01:00:38 on dist-test-slave-07c2"
server_key: "aaa0185df2210e75c84834f1d46551a3"
server_key_iv: "3d20a1b2e53a92796108134e76856df7"
server_key_version: "encryptionkey@0"
I20260809 01:00:38.964974 15680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444"
format_stamp: "Formatted at 2026-08-09 01:00:38 on dist-test-slave-07c2"
server_key: "aaa0185df2210e75c84834f1d46551a3"
server_key_iv: "3d20a1b2e53a92796108134e76856df7"
server_key_version: "encryptionkey@0"
I20260809 01:00:38.969915 15680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 01:00:38.973172 15696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:38.974372 15680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 01:00:38.974557 15680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444"
format_stamp: "Formatted at 2026-08-09 01:00:38 on dist-test-slave-07c2"
server_key: "aaa0185df2210e75c84834f1d46551a3"
server_key_iv: "3d20a1b2e53a92796108134e76856df7"
server_key_version: "encryptionkey@0"
I20260809 01:00:38.974749 15680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:38.988466 15680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:38.991993 15680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:38.992266 15680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:38.993093 15680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:38.994319 15680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:38.994418 15680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:38.994565 15680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:38.994638 15680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:39.028049 15680 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:45699
I20260809 01:00:39.028133 15815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:45699 every 8 connection(s)
I20260809 01:00:39.029336 15680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:00:39.031531 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 15680
I20260809 01:00:39.031761 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:00:39.032289 4426 external_mini_cluster.cc:1428] Setting key 808a3277d80b245fe2621edbfe4f7b89
Aug 09 01:00:39 dist-test-slave-07c2 krb5kdc[15424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237238, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:00:39.050729 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.031974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:59761 (local address 127.4.82.190:37893)
0809 01:00:39.032164 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:39.032169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:39.033434 (+ 1265us) server_negotiation.cc:407] Connection header received
0809 01:00:39.035376 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:39.035382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:39.035473 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:39.035578 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:39.036982 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.037689 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.039194 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.039814 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.039944 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.044407 (+ 4463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:39.044416 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:39.044424 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:39.044471 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:39.047415 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:39.048378 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:39.048386 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:39.048392 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:39.048569 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:39.049069 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:39.049072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:39.049074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:39.049333 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:39.049442 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:39.050327 (+ 885us) server_negotiation.cc:299] Negotiation successful
0809 01:00:39.050492 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 01:00:39.052284 15819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.032407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:37893 (local address 127.4.82.130:59761)
0809 01:00:39.033273 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:39.033326 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:39.035111 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:39.035805 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:39.035813 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:39.036290 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:39.036815 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.036832 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.037840 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.037850 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.039047 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.039059 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.040047 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.040057 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.040404 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.041152 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:39.041194 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:39.044171 (+ 2977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:39.047569 (+ 3398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:39.047580 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:39.047596 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:39.048235 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:39.048688 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:39.048695 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:39.048701 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:39.048953 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:39.049457 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:39.049467 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:39.049938 (+ 471us) client_negotiation.cc:769] Sending connection context
0809 01:00:39.050260 (+ 322us) client_negotiation.cc:240] Negotiation successful
0809 01:00:39.050794 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":318,"threads_started":1}
I20260809 01:00:39.053628 15817 heartbeater.cc:347] Connected to a master server at 127.4.82.190:37893
I20260809 01:00:39.053936 15817 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:39.054849 15817 heartbeater.cc:511] Master 127.4.82.190:37893 requested a full tablet report, sending...
I20260809 01:00:39.056440 15472 ts_manager.cc:194] Registered new tserver with Master: 0cd5f8dd73074c5496f5dc4ea3ac9444 (127.4.82.130:45699)
I20260809 01:00:39.057253 15472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:59761
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 01:00:39.093497 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37893
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:00:39.427882 15822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:00:39.428117 15822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:00:39.428293 15822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:00:39.435571 15822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:00:39.435706 15822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:00:39.435894 15822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:00:39.444921 15822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:00:39.445974 15822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:00:39.447186 15822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:00:39.459492 15827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:39.459540 15830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:00:39.459578 15828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:00:39.460546 15822 server_base.cc:1034] running on GCE node
I20260809 01:00:39.461042 15822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:00:39.461920 15822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:00:39.463063 15822 hybrid_clock.cc:648] HybridClock initialized: now 1786237239463019 us; error 34 us; skew 500 ppm
I20260809 01:00:39.463399 15822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:00:39 dist-test-slave-07c2 krb5kdc[15424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237239, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:00:39.466374 15822 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:00:39.467706 15822 webserver.cc:460] Webserver started at http://127.4.82.131:46149/ using document root <none> and password file <none>
I20260809 01:00:39.468303 15822 fs_manager.cc:359] Metadata directory not provided
I20260809 01:00:39.468438 15822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:00:39.468822 15822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:00:39.471657 15822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "943a13dc3b2e458ba4f41fa155068215"
format_stamp: "Formatted at 2026-08-09 01:00:39 on dist-test-slave-07c2"
server_key: "43971c6965f946f8e0601723a59e4fe9"
server_key_iv: "8c8a7b0b8171a172623469da1681a28a"
server_key_version: "encryptionkey@0"
I20260809 01:00:39.472381 15822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "943a13dc3b2e458ba4f41fa155068215"
format_stamp: "Formatted at 2026-08-09 01:00:39 on dist-test-slave-07c2"
server_key: "43971c6965f946f8e0601723a59e4fe9"
server_key_iv: "8c8a7b0b8171a172623469da1681a28a"
server_key_version: "encryptionkey@0"
I20260809 01:00:39.477221 15822 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 01:00:39.480594 15837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:39.481807 15822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260809 01:00:39.481968 15822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "943a13dc3b2e458ba4f41fa155068215"
format_stamp: "Formatted at 2026-08-09 01:00:39 on dist-test-slave-07c2"
server_key: "43971c6965f946f8e0601723a59e4fe9"
server_key_iv: "8c8a7b0b8171a172623469da1681a28a"
server_key_version: "encryptionkey@0"
I20260809 01:00:39.482157 15822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:00:39.506844 15822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:00:39.510432 15822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:00:39.510720 15822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:00:39.511611 15822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:00:39.512835 15822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:00:39.512935 15822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:39.513074 15822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:00:39.513151 15822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:00:39.546608 15822 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:34473
I20260809 01:00:39.546741 15959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:34473 every 8 connection(s)
I20260809 01:00:39.547976 15822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:00:39.555825 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 15822
I20260809 01:00:39.556022 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:00:39.556505 4426 external_mini_cluster.cc:1428] Setting key 69bd36434fd36cd2ca4a3d098fb465c3
I20260809 01:00:39.558000 15674 heartbeater.cc:503] Master 127.4.82.190:37893 was elected leader, sending a full tablet report...
Aug 09 01:00:39 dist-test-slave-07c2 krb5kdc[15424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237239, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:00:39.567950 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.550411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:35865 (local address 127.4.82.190:37893)
0809 01:00:39.550578 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:39.550582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:39.551662 (+ 1080us) server_negotiation.cc:407] Connection header received
0809 01:00:39.553485 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:39.553491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:39.553581 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:39.553686 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:39.555054 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.556155 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.557902 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.558580 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.558754 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.562815 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:39.562820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:39.562823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:39.562853 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:39.565428 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:39.566091 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:39.566094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:39.566096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:39.566226 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:39.566574 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:39.566577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:39.566579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:39.566832 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:39.566947 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:39.567551 (+ 604us) server_negotiation.cc:299] Negotiation successful
0809 01:00:39.567700 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 01:00:39.568809 15962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.550791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:37893 (local address 127.4.82.131:35865)
0809 01:00:39.551508 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:39.551545 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:39.553229 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:39.553904 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:39.553911 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:39.554374 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:00:39.554889 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.554905 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.556321 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.556325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.557684 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.557702 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.559007 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.559016 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.559356 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.560147 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:00:39.560168 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:00:39.562620 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:00:39.565560 (+ 2940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:39.565565 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:39.565575 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:39.565975 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:39.566314 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:00:39.566317 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:00:39.566319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:00:39.566486 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:00:39.566936 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:00:39.566942 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:00:39.567210 (+ 268us) client_negotiation.cc:769] Sending connection context
0809 01:00:39.567478 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 01:00:39.567813 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":261,"threads_started":1}
I20260809 01:00:39.570117 15960 heartbeater.cc:347] Connected to a master server at 127.4.82.190:37893
I20260809 01:00:39.570442 15960 heartbeater.cc:464] Registering TS with master...
I20260809 01:00:39.571197 15960 heartbeater.cc:511] Master 127.4.82.190:37893 requested a full tablet report, sending...
I20260809 01:00:39.572585 15471 ts_manager.cc:194] Registered new tserver with Master: 943a13dc3b2e458ba4f41fa155068215 (127.4.82.131:34473)
I20260809 01:00:39.573345 15471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:35865
I20260809 01:00:39.583696 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:00:39.599249 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.588210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.4.82.190:37893)
0809 01:00:39.588333 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:39.588337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:39.588807 (+ 470us) server_negotiation.cc:407] Connection header received
0809 01:00:39.588928 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:39.588931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:39.588986 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:39.589060 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:39.590336 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.591003 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.592202 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.592578 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.592672 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.594691 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:39.594695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:39.594698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:39.594726 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:39.597058 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:39.597648 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:39.597652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:39.597654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:39.597768 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:39.598053 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:39.598056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:39.598058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:39.598331 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:39.598501 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:39.598770 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 01:00:39.598912 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 01:00:39.603004 15471 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:00:39.610080 15471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:00:39.645226 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.622540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47384 (local address 127.4.82.131:34473)
0809 01:00:39.635013 (+ 12473us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:39.635024 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:39.635061 (+ 37us) server_negotiation.cc:407] Connection header received
0809 01:00:39.635172 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:39.635183 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:39.635502 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:39.636984 (+ 1482us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:39.637314 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.639262 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.642402 (+ 3140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.643887 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.644017 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.644237 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:39.644680 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 01:00:39.644856 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12255,"thread_start_us":12095,"threads_started":1}
I20260809 01:00:39.645325 15975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.622340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:34473 (local address 127.0.0.1:47384)
0809 01:00:39.634054 (+ 11714us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:39.634105 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:39.634300 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:39.635771 (+ 1471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:39.635781 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:39.635823 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:39.636752 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.636769 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.639456 (+ 2687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.639464 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.642255 (+ 2791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.642272 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.644100 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.644106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.644386 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.644488 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 01:00:39.644638 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 01:00:39.644864 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10392,"mutex_wait_us":42,"thread_start_us":13130,"threads_started":1}
I20260809 01:00:39.647483 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.622464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:45699 (local address 127.0.0.1:38276)
0809 01:00:39.635758 (+ 13294us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:39.635781 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:39.635896 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:39.640334 (+ 4438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:39.640339 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:39.640364 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:39.641141 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.641168 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.642887 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.642892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.644866 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.644880 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.646574 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.646585 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.646772 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.646802 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 01:00:39.646902 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 01:00:39.647026 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13144,"mutex_wait_us":14,"thread_start_us":18253,"threads_started":1}
I20260809 01:00:39.649094 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.622982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:46559 (local address 127.0.0.1:38116)
0809 01:00:39.641141 (+ 18159us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:39.641162 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:39.641289 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:39.641863 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:39.641868 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:39.641894 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:39.642679 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.642693 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.644379 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.644384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.646612 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.646628 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.648330 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.648338 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.648498 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.648528 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 01:00:39.648639 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 01:00:39.648764 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18010,"mutex_wait_us":42,"thread_start_us":10275,"threads_started":1}
I20260809 01:00:39.649149 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.623278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.4.82.129:46559)
0809 01:00:39.639514 (+ 16236us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:39.639522 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:39.641267 (+ 1745us) server_negotiation.cc:407] Connection header received
0809 01:00:39.641427 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:39.641434 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:39.641651 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:39.641854 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:39.642982 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.644208 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.646809 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.648119 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.648235 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.648384 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:39.648698 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 01:00:39.648841 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16022,"thread_start_us":15855,"threads_started":1}
I20260809 01:00:39.649145 15976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.623073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38276 (local address 127.4.82.130:45699)
0809 01:00:39.639886 (+ 16813us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:39.639892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:39.639913 (+ 21us) server_negotiation.cc:407] Connection header received
0809 01:00:39.639982 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:39.639986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:39.640130 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:39.640340 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:39.641347 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.642729 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.645065 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.646329 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.648449 (+ 2120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.648585 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:39.648668 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 01:00:39.648832 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16674,"thread_start_us":16601,"threads_started":1}
I20260809 01:00:39.649387 15890 tablet_service.cc:1467] Processing CreateTablet for tablet 4069e91b030249b09bdabbf02046eba8 (DEFAULT_TABLE table=test-table [id=e4df0a9f714e482fac02728e79e1a054]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:39.650782 15890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4069e91b030249b09bdabbf02046eba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:39.652752 15605 tablet_service.cc:1467] Processing CreateTablet for tablet 4069e91b030249b09bdabbf02046eba8 (DEFAULT_TABLE table=test-table [id=e4df0a9f714e482fac02728e79e1a054]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:39.652577 15745 tablet_service.cc:1467] Processing CreateTablet for tablet 4069e91b030249b09bdabbf02046eba8 (DEFAULT_TABLE table=test-table [id=e4df0a9f714e482fac02728e79e1a054]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:00:39.654049 15605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4069e91b030249b09bdabbf02046eba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:39.654424 15745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4069e91b030249b09bdabbf02046eba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:00:39.662832 15978 tablet_bootstrap.cc:492] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215: Bootstrap starting.
I20260809 01:00:39.666101 15978 tablet_bootstrap.cc:654] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:39.667097 15978 log.cc:824] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215: Log is configured to *not* fsync() on all Append() calls
I20260809 01:00:39.671751 15981 tablet_bootstrap.cc:492] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531: Bootstrap starting.
I20260809 01:00:39.676097 15981 tablet_bootstrap.cc:654] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:39.677994 15981 log.cc:824] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531: Log is configured to *not* fsync() on all Append() calls
I20260809 01:00:39.678279 15982 tablet_bootstrap.cc:492] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444: Bootstrap starting.
I20260809 01:00:39.679335 15978 tablet_bootstrap.cc:492] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215: No bootstrap required, opened a new log
I20260809 01:00:39.679709 15978 ts_tablet_manager.cc:1397] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.003s
I20260809 01:00:39.680830 15981 tablet_bootstrap.cc:492] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531: No bootstrap required, opened a new log
I20260809 01:00:39.681082 15981 ts_tablet_manager.cc:1397] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 01:00:39.681671 15982 tablet_bootstrap.cc:654] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444: Neither blocks nor log segments found. Creating new log.
I20260809 01:00:39.682874 15982 log.cc:824] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444: Log is configured to *not* fsync() on all Append() calls
I20260809 01:00:39.685722 15982 tablet_bootstrap.cc:492] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444: No bootstrap required, opened a new log
I20260809 01:00:39.685974 15982 ts_tablet_manager.cc:1397] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260809 01:00:39.689181 15981 raft_consensus.cc:348] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943a13dc3b2e458ba4f41fa155068215" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34473 } } peers { permanent_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45699 } } peers { permanent_uuid: "9c63f0389c624470b3bf105ff7e13531" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46559 } }
I20260809 01:00:39.689524 15981 raft_consensus.cc:374] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:39.689625 15981 raft_consensus.cc:729] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c63f0389c624470b3bf105ff7e13531, State: Initialized, Role: FOLLOWER
I20260809 01:00:39.690205 15981 consensus_queue.cc:260] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943a13dc3b2e458ba4f41fa155068215" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34473 } } peers { permanent_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45699 } } peers { permanent_uuid: "9c63f0389c624470b3bf105ff7e13531" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46559 } }
I20260809 01:00:39.690201 15978 raft_consensus.cc:348] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943a13dc3b2e458ba4f41fa155068215" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34473 } } peers { permanent_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45699 } } peers { permanent_uuid: "9c63f0389c624470b3bf105ff7e13531" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46559 } }
I20260809 01:00:39.690584 15978 raft_consensus.cc:374] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:39.690745 15978 raft_consensus.cc:729] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943a13dc3b2e458ba4f41fa155068215, State: Initialized, Role: FOLLOWER
I20260809 01:00:39.691710 15981 ts_tablet_manager.cc:1428] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 01:00:39.692624 15978 consensus_queue.cc:260] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943a13dc3b2e458ba4f41fa155068215" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34473 } } peers { permanent_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45699 } } peers { permanent_uuid: "9c63f0389c624470b3bf105ff7e13531" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46559 } }
I20260809 01:00:39.693696 15960 heartbeater.cc:503] Master 127.4.82.190:37893 was elected leader, sending a full tablet report...
I20260809 01:00:39.694249 15978 ts_tablet_manager.cc:1428] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260809 01:00:39.695957 15982 raft_consensus.cc:348] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943a13dc3b2e458ba4f41fa155068215" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34473 } } peers { permanent_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45699 } } peers { permanent_uuid: "9c63f0389c624470b3bf105ff7e13531" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46559 } }
I20260809 01:00:39.696390 15982 raft_consensus.cc:374] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:00:39.696493 15982 raft_consensus.cc:729] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd5f8dd73074c5496f5dc4ea3ac9444, State: Initialized, Role: FOLLOWER
I20260809 01:00:39.697021 15982 consensus_queue.cc:260] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943a13dc3b2e458ba4f41fa155068215" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34473 } } peers { permanent_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45699 } } peers { permanent_uuid: "9c63f0389c624470b3bf105ff7e13531" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46559 } }
I20260809 01:00:39.698117 15817 heartbeater.cc:503] Master 127.4.82.190:37893 was elected leader, sending a full tablet report...
I20260809 01:00:39.698458 15982 ts_tablet_manager.cc:1428] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260809 01:00:39.706471 15986 raft_consensus.cc:482] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:00:39.706704 15986 raft_consensus.cc:504] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943a13dc3b2e458ba4f41fa155068215" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34473 } } peers { permanent_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45699 } } peers { permanent_uuid: "9c63f0389c624470b3bf105ff7e13531" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46559 } }
I20260809 01:00:39.708250 15986 leader_election.cc:288] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cd5f8dd73074c5496f5dc4ea3ac9444 (127.4.82.130:45699), 9c63f0389c624470b3bf105ff7e13531 (127.4.82.129:46559)
I20260809 01:00:39.713413 15985 raft_consensus.cc:482] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:00:39.713690 15985 raft_consensus.cc:504] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943a13dc3b2e458ba4f41fa155068215" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34473 } } peers { permanent_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45699 } } peers { permanent_uuid: "9c63f0389c624470b3bf105ff7e13531" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46559 } }
I20260809 01:00:39.715309 15976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.708478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:41877 (local address 127.4.82.130:45699)
0809 01:00:39.708687 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:39.708696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:39.708975 (+ 279us) server_negotiation.cc:407] Connection header received
0809 01:00:39.709245 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:39.709251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:39.709326 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:39.709404 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:39.709829 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.710812 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.713194 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.714328 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.714482 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.714552 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:39.714885 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 01:00:39.715021 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 01:00:39.715368 15985 leader_election.cc:288] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 943a13dc3b2e458ba4f41fa155068215 (127.4.82.131:34473), 0cd5f8dd73074c5496f5dc4ea3ac9444 (127.4.82.130:45699)
I20260809 01:00:39.716707 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.708747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:40409 (local address 127.4.82.129:46559)
0809 01:00:39.708946 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:39.708953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:39.709531 (+ 578us) server_negotiation.cc:407] Connection header received
0809 01:00:39.709665 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:39.709669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:39.709725 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:39.709818 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:39.710280 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.711170 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.713602 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.714910 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.715045 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.715110 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:39.716236 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0809 01:00:39.716403 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 01:00:39.715263 15962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.708611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:45699 (local address 127.4.82.131:41877)
0809 01:00:39.708883 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:39.708899 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:39.709049 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:39.709457 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:39.709464 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:39.709497 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:39.709710 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.709718 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.710936 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.710939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.713023 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.713038 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.714491 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.714497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.714691 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.714725 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 01:00:39.714824 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 01:00:39.714965 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":58}
I20260809 01:00:39.717494 15989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.708737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:46559 (local address 127.4.82.131:40409)
0809 01:00:39.709426 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:39.709442 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:39.709561 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:39.709845 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:39.709848 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:39.709867 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:39.710144 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.710150 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.711323 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.711328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.713440 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.713456 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.715256 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.715262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.715476 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.715520 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 01:00:39.715666 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 01:00:39.715803 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":37,"thread_start_us":411,"threads_started":1}
I20260809 01:00:39.717712 15768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4069e91b030249b09bdabbf02046eba8" candidate_uuid: "943a13dc3b2e458ba4f41fa155068215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" is_pre_election: true
I20260809 01:00:39.718323 15768 raft_consensus.cc:2454] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 943a13dc3b2e458ba4f41fa155068215 in term 0.
I20260809 01:00:39.719007 15839 leader_election.cc:302] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cd5f8dd73074c5496f5dc4ea3ac9444, 943a13dc3b2e458ba4f41fa155068215; no voters:
I20260809 01:00:39.719619 15986 raft_consensus.cc:2781] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:00:39.719830 15986 raft_consensus.cc:482] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:00:39.720014 15986 raft_consensus.cc:3037] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:39.719920 15625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4069e91b030249b09bdabbf02046eba8" candidate_uuid: "943a13dc3b2e458ba4f41fa155068215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c63f0389c624470b3bf105ff7e13531" is_pre_election: true
I20260809 01:00:39.720384 15625 raft_consensus.cc:2454] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 943a13dc3b2e458ba4f41fa155068215 in term 0.
I20260809 01:00:39.723461 15986 raft_consensus.cc:504] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943a13dc3b2e458ba4f41fa155068215" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34473 } } peers { permanent_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45699 } } peers { permanent_uuid: "9c63f0389c624470b3bf105ff7e13531" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46559 } }
I20260809 01:00:39.724278 15986 leader_election.cc:288] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [CANDIDATE]: Term 1 election: Requested vote from peers 0cd5f8dd73074c5496f5dc4ea3ac9444 (127.4.82.130:45699), 9c63f0389c624470b3bf105ff7e13531 (127.4.82.129:46559)
I20260809 01:00:39.724570 15768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4069e91b030249b09bdabbf02046eba8" candidate_uuid: "943a13dc3b2e458ba4f41fa155068215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444"
I20260809 01:00:39.724776 15768 raft_consensus.cc:3037] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:39.724861 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.715955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:33635 (local address 127.4.82.131:34473)
0809 01:00:39.716156 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:39.716164 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:39.718624 (+ 2460us) server_negotiation.cc:407] Connection header received
0809 01:00:39.718920 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:39.718925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:39.719000 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:39.719088 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:39.719961 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.721092 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.722811 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.723798 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.723955 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.724025 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:39.724483 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 01:00:39.724617 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 01:00:39.725646 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.715776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:34473 (local address 127.4.82.129:33635)
0809 01:00:39.718513 (+ 2737us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:39.718536 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:39.718752 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:39.719260 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:39.719264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:39.719316 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:39.719771 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.719784 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.721282 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.721287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.722657 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.722670 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.724018 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.724025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.724262 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.724297 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 01:00:39.725069 (+ 772us) client_negotiation.cc:240] Negotiation successful
0809 01:00:39.725205 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2579,"mutex_wait_us":24,"thread_start_us":3262,"threads_started":1}
I20260809 01:00:39.726121 15625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4069e91b030249b09bdabbf02046eba8" candidate_uuid: "943a13dc3b2e458ba4f41fa155068215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c63f0389c624470b3bf105ff7e13531"
I20260809 01:00:39.726325 15625 raft_consensus.cc:3037] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:00:39.726504 15976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.716333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:46895 (local address 127.4.82.130:45699)
0809 01:00:39.716460 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:39.716466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:39.719385 (+ 2919us) server_negotiation.cc:407] Connection header received
0809 01:00:39.719660 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:39.719664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:39.719746 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:39.719811 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:00:39.720732 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.721791 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.724231 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.725453 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.725655 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.725755 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:39.726091 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 01:00:39.726230 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 01:00:39.726691 15911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4069e91b030249b09bdabbf02046eba8" candidate_uuid: "9c63f0389c624470b3bf105ff7e13531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943a13dc3b2e458ba4f41fa155068215" is_pre_election: true
I20260809 01:00:39.727164 15911 raft_consensus.cc:2379] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c63f0389c624470b3bf105ff7e13531 in current term 1: Already voted for candidate 943a13dc3b2e458ba4f41fa155068215 in this term.
I20260809 01:00:39.727180 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.715825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:45699 (local address 127.4.82.129:46895)
0809 01:00:39.719257 (+ 3432us) negotiation.cc:107] Waiting for socket to connect
0809 01:00:39.719277 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:00:39.719492 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:00:39.720017 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:00:39.720022 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:00:39.720054 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:00:39.720524 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.720543 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.721976 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.721981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.724065 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:00:39.724081 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.725718 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.725726 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:00:39.725895 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.725929 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 01:00:39.726653 (+ 724us) client_negotiation.cc:240] Negotiation successful
0809 01:00:39.726787 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3319,"mutex_wait_us":1,"thread_start_us":2268,"threads_started":1}
I20260809 01:00:39.727969 15767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4069e91b030249b09bdabbf02046eba8" candidate_uuid: "9c63f0389c624470b3bf105ff7e13531" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" is_pre_election: true
I20260809 01:00:39.728333 15768 raft_consensus.cc:2454] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 943a13dc3b2e458ba4f41fa155068215 in term 1.
I20260809 01:00:39.728729 15839 leader_election.cc:302] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cd5f8dd73074c5496f5dc4ea3ac9444, 943a13dc3b2e458ba4f41fa155068215; no voters:
I20260809 01:00:39.728997 15986 raft_consensus.cc:2781] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:00:39.728921 15556 leader_election.cc:302] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c63f0389c624470b3bf105ff7e13531; no voters: 0cd5f8dd73074c5496f5dc4ea3ac9444, 943a13dc3b2e458ba4f41fa155068215
I20260809 01:00:39.729653 15986 raft_consensus.cc:686] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 1 LEADER]: Becoming Leader. State: Replica: 943a13dc3b2e458ba4f41fa155068215, State: Running, Role: LEADER
I20260809 01:00:39.730118 15986 consensus_queue.cc:237] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943a13dc3b2e458ba4f41fa155068215" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34473 } } peers { permanent_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45699 } } peers { permanent_uuid: "9c63f0389c624470b3bf105ff7e13531" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46559 } }
I20260809 01:00:39.730253 15625 raft_consensus.cc:2454] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 943a13dc3b2e458ba4f41fa155068215 in term 1.
I20260809 01:00:39.730453 15985 raft_consensus.cc:2726] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:00:39.734776 15471 catalog_manager.cc:5495] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 reported cstate change: term changed from 0 to 1, leader changed from <none> to 943a13dc3b2e458ba4f41fa155068215 (127.4.82.131). New cstate: current_term: 1 leader_uuid: "943a13dc3b2e458ba4f41fa155068215" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943a13dc3b2e458ba4f41fa155068215" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c63f0389c624470b3bf105ff7e13531" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46559 } health_report { overall_health: UNKNOWN } } }
I20260809 01:00:39.771034 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.766122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.4.82.131:34473)
0809 01:00:39.766298 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:39.766302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:39.766319 (+ 17us) server_negotiation.cc:407] Connection header received
0809 01:00:39.766503 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:39.766506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:39.766599 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:39.766719 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:00:39.767033 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.767831 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.769111 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.769485 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.769575 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.770007 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:00:39.770493 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:00:39.770593 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:39.770680 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 01:00:39.770735 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 01:00:39.777066 15675 tablet.cc:2369] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:00:39.778265 15767 raft_consensus.cc:1261] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444 [term 1 FOLLOWER]: Refusing update from remote peer 943a13dc3b2e458ba4f41fa155068215: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:39.778288 15625 raft_consensus.cc:1261] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [term 1 FOLLOWER]: Refusing update from remote peer 943a13dc3b2e458ba4f41fa155068215: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:00:39.779114 15986 consensus_queue.cc:1035] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd5f8dd73074c5496f5dc4ea3ac9444" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:00:39.779453 15992 consensus_queue.cc:1035] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c63f0389c624470b3bf105ff7e13531" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 01:00:39.781152 15818 tablet.cc:2369] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:00:39.790710 15995 mvcc.cc:205] Tried to move back new op lower bound from 7316427734122827776 to 7316427733939179520. Current Snapshot: MvccSnapshot[applied={T|T < 7316427734122827776}]
I20260809 01:00:39.803846 15472 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:00:39.804073 15472 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:00:39.807408 15472 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edd4c13d465546b2a8947eb7282a8605: Sending DeleteTablet for 3 replicas of tablet 4069e91b030249b09bdabbf02046eba8
I20260809 01:00:39.808418 15890 tablet_service.cc:1514] Processing DeleteTablet for tablet 4069e91b030249b09bdabbf02046eba8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:39 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:47384
I20260809 01:00:39.808444 15745 tablet_service.cc:1514] Processing DeleteTablet for tablet 4069e91b030249b09bdabbf02046eba8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:39 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:38276
I20260809 01:00:39.808786 15605 tablet_service.cc:1514] Processing DeleteTablet for tablet 4069e91b030249b09bdabbf02046eba8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:00:39 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:38116
I20260809 01:00:39.809301 16005 tablet_replica.cc:330] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215: stopping tablet replica
I20260809 01:00:39.809679 16005 raft_consensus.cc:2229] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:00:39.810109 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 15537
I20260809 01:00:39.810814 16004 tablet_replica.cc:330] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444: stopping tablet replica
I20260809 01:00:39.811107 16006 tablet_replica.cc:330] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531: stopping tablet replica
I20260809 01:00:39.811524 16006 raft_consensus.cc:2229] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:00:39.811990 16006 raft_consensus.cc:2258] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:39.810302 16005 raft_consensus.cc:2258] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:39.813055 16004 raft_consensus.cc:2229] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:00:39.813607 16004 raft_consensus.cc:2258] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:00:39.814495 16005 ts_tablet_manager.cc:1905] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:39.816062 16004 ts_tablet_manager.cc:1905] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:39.820391 16006 ts_tablet_manager.cc:1905] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:00:39.824640 16004 ts_tablet_manager.cc:1918] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:00:39.824877 16004 log.cc:1196] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/4069e91b030249b09bdabbf02046eba8
I20260809 01:00:39.825522 16004 ts_tablet_manager.cc:1939] T 4069e91b030249b09bdabbf02046eba8 P 0cd5f8dd73074c5496f5dc4ea3ac9444: Deleting consensus metadata
I20260809 01:00:39.827469 15455 catalog_manager.cc:4841] TS 0cd5f8dd73074c5496f5dc4ea3ac9444 (127.4.82.130:45699): tablet 4069e91b030249b09bdabbf02046eba8 (table test-table [id=e4df0a9f714e482fac02728e79e1a054]) successfully deleted
I20260809 01:00:39.828119 16006 ts_tablet_manager.cc:1918] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:00:39.828325 16006 log.cc:1196] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/4069e91b030249b09bdabbf02046eba8
I20260809 01:00:39.828964 16006 ts_tablet_manager.cc:1939] T 4069e91b030249b09bdabbf02046eba8 P 9c63f0389c624470b3bf105ff7e13531: Deleting consensus metadata
I20260809 01:00:39.830153 16005 ts_tablet_manager.cc:1918] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:00:39.830650 15458 catalog_manager.cc:4841] TS 9c63f0389c624470b3bf105ff7e13531 (127.4.82.129:46559): tablet 4069e91b030249b09bdabbf02046eba8 (table test-table [id=e4df0a9f714e482fac02728e79e1a054]) successfully deleted
I20260809 01:00:39.830765 16005 log.cc:1196] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/4069e91b030249b09bdabbf02046eba8
I20260809 01:00:39.831422 16005 ts_tablet_manager.cc:1939] T 4069e91b030249b09bdabbf02046eba8 P 943a13dc3b2e458ba4f41fa155068215: Deleting consensus metadata
Aug 09 01:00:39 dist-test-slave-07c2 krb5kdc[15424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 01:00:39.833078 15454 catalog_manager.cc:4841] TS 943a13dc3b2e458ba4f41fa155068215 (127.4.82.131:34473): tablet 4069e91b030249b09bdabbf02046eba8 (table test-table [id=e4df0a9f714e482fac02728e79e1a054]) successfully deleted
I20260809 01:00:39.839699 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.811215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38130 (local address 127.4.82.129:46559)
0809 01:00:39.811450 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:39.811457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:39.823199 (+ 11742us) server_negotiation.cc:407] Connection header received
0809 01:00:39.823714 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:39.823719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:39.823798 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:39.823874 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:39.825777 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.826898 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.828862 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.829519 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.829616 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.834135 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:39.834145 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:39.834158 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:39.834189 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:39.836933 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:39.837604 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:39.837610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:39.837612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:39.837735 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:39.838118 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:39.838121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:39.838123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:39.838470 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:39.838779 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:39.839081 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 01:00:39.839354 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 01:00:39.840322 15669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:39.942613 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 15537
I20260809 01:00:39.963868 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 15680
Aug 09 01:00:39 dist-test-slave-07c2 krb5kdc[15424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 01:00:39.981060 15976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:39.964728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38282 (local address 127.4.82.130:45699)
0809 01:00:39.964965 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:39.964971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:39.964994 (+ 23us) server_negotiation.cc:407] Connection header received
0809 01:00:39.965128 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:39.965134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:39.965231 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:39.965342 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:39.966522 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.967692 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.968911 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:39.969582 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:39.969716 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:39.974139 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:39.974168 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:39.974189 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:39.974251 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:39.978089 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:39.978796 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:39.978807 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:39.978812 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:39.978972 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:39.979391 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:39.979401 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:39.979404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:39.979849 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:39.980160 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:39.980490 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 01:00:39.980731 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 01:00:39.981678 15810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:40.088387 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 15680
I20260809 01:00:40.106897 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 15822
Aug 09 01:00:40 dist-test-slave-07c2 krb5kdc[15424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 01:00:40.124608 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:00:40.107794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.4.82.131:34473)
0809 01:00:40.107976 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 01:00:40.107980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:00:40.108000 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:00:40.108257 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:00:40.108260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:00:40.108323 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:00:40.108417 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:00:40.110202 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:40.110923 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:40.112899 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:00:40.113333 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:00:40.113427 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:00:40.118702 (+ 5275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:00:40.118720 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:00:40.118735 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:00:40.118789 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:00:40.121864 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:40.122510 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:40.122517 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:40.122519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:40.122639 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:00:40.123012 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:00:40.123018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:00:40.123021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:00:40.123479 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:00:40.123817 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 01:00:40.124068 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 01:00:40.124312 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 01:00:40.125252 15954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:40.251472 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 15822
I20260809 01:00:40.274789 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 15437
I20260809 01:00:40.276010 15505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:00:40.384809 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 15437
2026-08-09T01:00:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4288 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:00:40 dist-test-slave-07c2 krb5kdc[16014](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:00:40 dist-test-slave-07c2 krb5kdc[16014](info): set up 2 sockets
Aug 09 01:00:40 dist-test-slave-07c2 krb5kdc[16014](info): commencing operation
krb5kdc: starting...
W20260809 01:00:41.531957 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:00:41 dist-test-slave-07c2 krb5kdc[16014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:00:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:00:41Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.82.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.82.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/rangeradmin_127.4.82.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/rangeradmin_127.4.82.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.82.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.82.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/rangerlookup_127.4.82.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/rangerlookup_127.4.82.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.82.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.82.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/HTTP_127.4.82.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/HTTP_127.4.82.148@KRBTEST.COM.keytab.
I20260809 01:00:41.888082 4426 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskua6Q0e/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/postgres -l logfile start
2026-08-09 01:00:44.277 UTC [16047] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 01:00:44.277 UTC [16047] LOG: listening on IPv4 address "127.4.82.148", port 44827
2026-08-09 01:00:44.286 UTC [16047] LOG: listening on Unix socket "/tmp/.s.PGSQL.44827"
2026-08-09 01:00:44.307 UTC [16050] LOG: database system was shut down at 2026-08-09 01:00:43 UTC
2026-08-09 01:00:44.311 UTC [16047] LOG: database system is ready to accept connections
I20260809 01:00:45.307091 4426 mini_postgres.cc:96] Postgres bound to 44827
2026-08-09 01:00:45.331 UTC [16058] FATAL: database "slave" does not exist
127.4.82.148:44827 - accepting connections
I20260809 01:00:45.332804 4426 mini_ranger.cc:161] Starting Ranger...
I20260809 01:00:45.362584 4426 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 01:00:45.808880 4426 mini_ranger.cc:87] Created ranger Postgres database
I20260809 01:00:45.809026 4426 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-admin
2026-08-09 01:00:46,187 [I] DB FLAVOR :POSTGRES
2026-08-09 01:00:46,188 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 01:00:46,188 [I] Checking connection..
2026-08-09 01:00:46,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 01:00:46,458 [I] Checking connection passed.
2026-08-09 01:00:46,459 [I] --------- Verifying version history table ---------
2026-08-09 01:00:46,459 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 01:00:46,729 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 01:00:46,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 01:00:46,983 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 01:00:46,984 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 01:00:46,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 01:00:47.230 UTC [16125] WARNING: there is no transaction in progress
2026-08-09 01:00:47,237 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 01:00:47,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 01:00:47,497 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 01:00:47,497 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 01:00:47,497 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 01:00:47,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 01:00:47,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-07c2.c.gcp-upstream.internal','N') ;"
2026-08-09 01:00:48,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 01:00:48,492 [I] Table x_portal_user does not exist in database ranger
2026-08-09 01:00:48,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 01:00:48,744 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 01:00:48,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 01:00:48,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 01:00:49,239 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 01:00:49,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 01:00:50.157 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.172 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.186 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.201 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.215 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.232 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.287 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.293 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.305 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.314 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.323 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.336 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.345 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.351 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.357 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50.363 UTC [16236] WARNING: there is no transaction in progress
2026-08-09 01:00:50,939 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 01:00:50,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 01:00:51,177 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 01:00:51,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 01:00:51,410 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 01:00:51,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 01:00:51,668 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 01:00:51,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 01:00:51,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 01:00:52,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-07c2.c.gcp-upstream.internal';"
2026-08-09 01:00:52,407 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 01:00:52,407 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 01:00:52,664 [I] DB_PATCHES have already been applied
I20260809 01:00:52.670524 4426 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-admin:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskua6Q0e/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 01:00:52.670642 4426 mini_ranger.cc:193] Using host: 127.4.82.148
I20260809 01:00:52.690819 4426 mini_ranger.cc:227] Ranger admin URL: http://127.4.82.148:46379
Aug 09, 2026 1:00:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 1:00:53 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 1:00:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 1:00:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 1:00:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 1:00:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 1:00:53 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 01:00:53.849184 4426 mini_ranger.cc:160] Time spent starting Ranger: real 8.516s user 0.002s sys 0.051s
Aug 09, 2026 1:01:13 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 1:01:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 1:01:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 1:01:13 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 1:01:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 1:01:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 1:01:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 1:01:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 1:01:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 1:01:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 1:01:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 1:01:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 1:01:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 1:01:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 1:01:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 1:01:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 1:01:13 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 1:01:14 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 01:01:15.201244 4426 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.82.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.82.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/rangerkms_127.4.82.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/rangerkms_127.4.82.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.82.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.82.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.82.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/HTTP_127.4.82.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.82.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/HTTP_127.4.82.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 01:01:15 dist-test-slave-07c2 krb5kdc[16014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237275, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 01:01:15.406973 4426 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 01:01:15.433745 4426 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 01:01:16.200219 4426 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 01:01:16.200390 4426 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-kms
I20260809 01:01:16.200500 4426 mini_ranger_kms.cc:227] Using postgres at 127.4.82.148:44827
2026-08-09 01:01:16,691 [I] DB FLAVOR :POSTGRES
2026-08-09 01:01:16,692 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 01:01:16,692 [I] Checking connection
2026-08-09 01:01:16,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 01:01:16,927 [I] connection success
2026-08-09 01:01:16,928 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 01:01:16,928 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 01:01:16,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 01:01:17,182 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 01:01:17,183 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 01:01:17,183 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 01:01:17,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:44827/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 01:01:17,438 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 01:01:17.869150 4426 mini_ranger_kms.cc:310] Created kms service
I20260809 01:01:18.057067 4426 mini_ranger_kms.cc:326] Created kudu user
I20260809 01:01:18.101732 4426 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 01:01:18 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237275, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:01:18.469180 4426 mini_ranger_kms.cc:384] Added ranger policy
I20260809 01:01:18.470614 4426 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-kms:/tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 01:01:18.470691 4426 mini_ranger_kms.cc:240] Using host: 127.4.82.148
I20260809 01:01:18.499373 4426 mini_ranger_kms.cc:276] Ranger KMS PID: 16436
I20260809 01:01:18.499544 4426 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.82.148:46913
Aug 09, 2026 1:01:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 1:01:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 1:01:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 1:01:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 1:01:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 1:01:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 01:01:20 dist-test-slave-07c2 krb5kdc[16014](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786237280, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.82.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:01:20.902979 4426 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.496s user 0.128s sys 0.397s
I20260809 01:01:20.903379 4426 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 01:01:20.903618 4426 mini_ranger_kms.cc:401] 127.4.82.148:46913/kms/v1/keys
Aug 09 01:01:21 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237280, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.82.148@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
Aug 09 01:01:21 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237280, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.82.148@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
Aug 09 01:01:24 dist-test-slave-07c2 krb5kdc[16014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:01:24.337178 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34935
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:34935
--ranger_config_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.82.148:46913/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:01:24.687860 16488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:01:24.688175 16488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:01:24.688341 16488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:01:24.695724 16488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:01:24.695843 16488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:01:24.695930 16488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:01:24.696033 16488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:01:24.696120 16488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:01:24.706254 16488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.82.148:46913/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:34935
--ranger_config_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:01:24.707532 16488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:01:24.708822 16488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:01:24.721202 16497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:01:24.721294 16494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:01:24.721289 16495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:01:24.722131 16488 server_base.cc:1034] running on GCE node
I20260809 01:01:24.722923 16488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:01:24.724361 16488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:01:24.725593 16488 hybrid_clock.cc:648] HybridClock initialized: now 1786237284725558 us; error 50 us; skew 500 ppm
I20260809 01:01:24.725935 16488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:01:24 dist-test-slave-07c2 krb5kdc[16014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237284, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:01:24.732154 16488 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:01:24.733623 16488 webserver.cc:460] Webserver started at http://127.4.82.190:42539/ using document root <none> and password file <none>
I20260809 01:01:24.734210 16488 fs_manager.cc:359] Metadata directory not provided
I20260809 01:01:24.734308 16488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:01:24.734555 16488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 01:01:24 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237284, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:01:24.794273 16488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "fc7a9977ab3d48a19ef39e2b61378496"
format_stamp: "Formatted at 2026-08-09 01:01:24 on dist-test-slave-07c2"
server_key: "0019fc77d229cc01a69541aa689600ce"
server_key_iv: "2baa8474f4e83f69f9025eba84014017"
server_key_version: "kuduclusterkey@0"
I20260809 01:01:24.795635 16488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "fc7a9977ab3d48a19ef39e2b61378496"
format_stamp: "Formatted at 2026-08-09 01:01:24 on dist-test-slave-07c2"
server_key: "0019fc77d229cc01a69541aa689600ce"
server_key_iv: "2baa8474f4e83f69f9025eba84014017"
server_key_version: "kuduclusterkey@0"
I20260809 01:01:24.802841 16488 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260809 01:01:24.819689 16504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:01:24.821184 16488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 01:01:24.821393 16488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "fc7a9977ab3d48a19ef39e2b61378496"
format_stamp: "Formatted at 2026-08-09 01:01:24 on dist-test-slave-07c2"
server_key: "0019fc77d229cc01a69541aa689600ce"
server_key_iv: "2baa8474f4e83f69f9025eba84014017"
server_key_version: "kuduclusterkey@0"
I20260809 01:01:24.821621 16488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:01:24.835130 16488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:01:24.838879 16488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:01:24.839237 16488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:01:24.860003 16488 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:34935
I20260809 01:01:24.860020 16559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:34935 every 8 connection(s)
I20260809 01:01:24.861331 16488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:01:24.865183 16560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:01:24.868455 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 16488
I20260809 01:01:24.868716 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:01:24.869462 4426 external_mini_cluster.cc:1428] Setting key 2a33d65df803e62b8cbf6b8042bc2ae4
I20260809 01:01:24.878402 16560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496: Bootstrap starting.
I20260809 01:01:24.881875 16560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496: Neither blocks nor log segments found. Creating new log.
I20260809 01:01:24.882846 16560 log.cc:824] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496: Log is configured to *not* fsync() on all Append() calls
Aug 09 01:01:24 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:01:24.886199 16560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496: No bootstrap required, opened a new log
I20260809 01:01:24.893523 16563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:24.871490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38448 (local address 127.4.82.190:34935)
0809 01:01:24.872309 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:24.872326 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:24.872377 (+ 51us) server_negotiation.cc:407] Connection header received
0809 01:01:24.874140 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:24.874169 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:24.874670 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:24.875093 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:01:24.876281 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:24.878182 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:24.879754 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:24.880811 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:24.881000 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:24.884811 (+ 3811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:01:24.884822 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:01:24.884840 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:01:24.884885 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:01:24.888861 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:24.889542 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:24.889551 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:24.889559 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:24.889782 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:24.890145 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:24.890149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:24.890153 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:24.890885 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:01:24.891186 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:24.891709 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 01:01:24.892148 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":354,"threads_started":1}
I20260809 01:01:24.896906 16560 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a9977ab3d48a19ef39e2b61378496" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34935 } }
I20260809 01:01:24.897228 16560 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:01:24.897305 16560 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc7a9977ab3d48a19ef39e2b61378496, State: Initialized, Role: FOLLOWER
I20260809 01:01:24.897758 16560 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a9977ab3d48a19ef39e2b61378496" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34935 } }
I20260809 01:01:24.897965 16560 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:01:24.898061 16560 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:01:24.898193 16560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:01:24.900370 16560 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a9977ab3d48a19ef39e2b61378496" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34935 } }
I20260809 01:01:24.900806 16560 leader_election.cc:302] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc7a9977ab3d48a19ef39e2b61378496; no voters:
I20260809 01:01:24.901523 16560 leader_election.cc:288] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:01:24.901846 16565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:01:24.903607 16565 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [term 1 LEADER]: Becoming Leader. State: Replica: fc7a9977ab3d48a19ef39e2b61378496, State: Running, Role: LEADER
I20260809 01:01:24.904345 16565 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a9977ab3d48a19ef39e2b61378496" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34935 } }
I20260809 01:01:24.904516 16560 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:01:24.907007 16568 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc7a9977ab3d48a19ef39e2b61378496. Latest consensus state: current_term: 1 leader_uuid: "fc7a9977ab3d48a19ef39e2b61378496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a9977ab3d48a19ef39e2b61378496" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34935 } } }
I20260809 01:01:24.907487 16566 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc7a9977ab3d48a19ef39e2b61378496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a9977ab3d48a19ef39e2b61378496" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34935 } } }
I20260809 01:01:24.907806 16568 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [sys.catalog]: This master's current role is: LEADER
I20260809 01:01:24.907807 16566 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496 [sys.catalog]: This master's current role is: LEADER
I20260809 01:01:24.912632 16573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:01:24.917999 16573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:01:24.924265 16560 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 01:01:24.940991 16573 catalog_manager.cc:1347] Generated new cluster ID: 5eca5cdfaefa4806a100789936d18567
I20260809 01:01:24.941183 16573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:01:24.964043 16573 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:01:24.966411 16573 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:01:24.981972 16573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496: Generated new TSK 0
I20260809 01:01:24.983202 16573 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 01:01:25.805845 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.82.148:46913/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34935
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 01:01:26 dist-test-slave-07c2 krb5kdc[16014](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786237286, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 01:01:26.415916 16606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:01:26.416446 16606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:01:26.416668 16606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:01:26.431425 16606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:01:26.431602 16606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:01:26.431860 16606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:01:26.447063 16606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.82.148:46913/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:01:26.448490 16606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:01:26.449926 16606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:01:26.466284 16617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:01:26.466567 16616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:01:26.466296 16619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:01:26.467880 16606 server_base.cc:1034] running on GCE node
I20260809 01:01:26.468832 16606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:01:26.470278 16606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:01:26.471565 16606 hybrid_clock.cc:648] HybridClock initialized: now 1786237286471464 us; error 52 us; skew 500 ppm
I20260809 01:01:26.472110 16606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:01:26 dist-test-slave-07c2 krb5kdc[16014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237286, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:01:26.477356 16606 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:01:26.479452 16606 webserver.cc:460] Webserver started at http://127.4.82.129:37897/ using document root <none> and password file <none>
I20260809 01:01:26.480217 16606 fs_manager.cc:359] Metadata directory not provided
I20260809 01:01:26.480381 16606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:01:26.480790 16606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 01:01:26 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237286, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:01:26.513264 16606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "96ae17813b7f403e8397237a4e24233d"
format_stamp: "Formatted at 2026-08-09 01:01:26 on dist-test-slave-07c2"
server_key: "335004b5b756b62e37d70c0ccc31ca65"
server_key_iv: "84895dff6189bfb45726d20be77ce4ba"
server_key_version: "kuduclusterkey@0"
I20260809 01:01:26.514664 16606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "96ae17813b7f403e8397237a4e24233d"
format_stamp: "Formatted at 2026-08-09 01:01:26 on dist-test-slave-07c2"
server_key: "335004b5b756b62e37d70c0ccc31ca65"
server_key_iv: "84895dff6189bfb45726d20be77ce4ba"
server_key_version: "kuduclusterkey@0"
I20260809 01:01:26.524675 16606 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.000s
I20260809 01:01:26.564478 16626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:01:26.566746 16606 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.007s sys 0.007s
I20260809 01:01:26.567028 16606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "96ae17813b7f403e8397237a4e24233d"
format_stamp: "Formatted at 2026-08-09 01:01:26 on dist-test-slave-07c2"
server_key: "335004b5b756b62e37d70c0ccc31ca65"
server_key_iv: "84895dff6189bfb45726d20be77ce4ba"
server_key_version: "kuduclusterkey@0"
I20260809 01:01:26.567463 16606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:01:26.618768 16606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:01:26.624799 16606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:01:26.625304 16606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:01:26.627547 16606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:01:26.629879 16606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:01:26.629988 16606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:01:26.630196 16606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:01:26.630297 16606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 01:01:26 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237286, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:01:26.842530 16606 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:38193
I20260809 01:01:26.842770 16748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:38193 every 8 connection(s)
I20260809 01:01:26.844288 16606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:01:26.853145 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 16606
I20260809 01:01:26.853371 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:01:26.854161 4426 external_mini_cluster.cc:1428] Setting key 197a2e9f9d7c9c041dfd2626e61be04f
Aug 09 01:01:26 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237286, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:01:26.881093 16751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:26.847359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:40635 (local address 127.4.82.190:34935)
0809 01:01:26.847935 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:26.847939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:26.849022 (+ 1083us) server_negotiation.cc:407] Connection header received
0809 01:01:26.850858 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:26.850863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:26.850970 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:26.851099 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:01:26.852622 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:26.853847 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:26.859403 (+ 5556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:26.860320 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:26.860473 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:26.866477 (+ 6004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:01:26.866487 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:01:26.866494 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:01:26.866568 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:01:26.875366 (+ 8798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:26.876730 (+ 1364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:26.876737 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:26.876750 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:26.877070 (+ 320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:26.878063 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:26.878068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:26.878071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:26.878700 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:01:26.878876 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:26.880077 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0809 01:01:26.880412 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":322,"threads_started":1}
I20260809 01:01:26.882402 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:26.847962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34935 (local address 127.4.82.129:40635)
0809 01:01:26.848849 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:26.848895 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 01:01:26.850574 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:01:26.851400 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:01:26.851408 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:01:26.851852 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:01:26.852430 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:26.852448 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:26.854034 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:26.854040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:26.856008 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:26.856025 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:26.861278 (+ 5253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:26.861292 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:26.861725 (+ 433us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:26.862738 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:01:26.862777 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:01:26.866205 (+ 3428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:01:26.875551 (+ 9346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:01:26.875586 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:01:26.875605 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:01:26.876534 (+ 929us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:01:26.877308 (+ 774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:01:26.877313 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:01:26.877348 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:01:26.877747 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:01:26.878931 (+ 1184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:01:26.878942 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:01:26.879491 (+ 549us) client_negotiation.cc:769] Sending connection context
0809 01:01:26.879867 (+ 376us) client_negotiation.cc:240] Negotiation successful
0809 01:01:26.880614 (+ 747us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":442,"threads_started":1}
I20260809 01:01:26.888860 16749 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34935
I20260809 01:01:26.889317 16749 heartbeater.cc:464] Registering TS with master...
I20260809 01:01:26.890506 16749 heartbeater.cc:511] Master 127.4.82.190:34935 requested a full tablet report, sending...
I20260809 01:01:26.893606 16521 ts_manager.cc:194] Registered new tserver with Master: 96ae17813b7f403e8397237a4e24233d (127.4.82.129:38193)
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
I20260809 01:01:26.895861 16521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:40635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 01:01:26.929281 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.82.148:46913/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34935
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 01:01:27 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237286, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
W20260809 01:01:27.620431 16755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:01:27.620850 16755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:01:27.621104 16755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:01:27.631134 16755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:01:27.631362 16755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:01:27.631619 16755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:01:27.644235 16755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.82.148:46913/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:01:27.645658 16755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:01:27.647378 16755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:01:27.669124 16773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:01:27.669260 16771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:01:27.671680 16755 server_base.cc:1034] running on GCE node
W20260809 01:01:27.671970 16770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:01:27.672940 16755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:01:27.674520 16755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:01:27.675812 16755 hybrid_clock.cc:648] HybridClock initialized: now 1786237287675691 us; error 68 us; skew 500 ppm
I20260809 01:01:27.676193 16755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:01:27 dist-test-slave-07c2 krb5kdc[16014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237287, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:01:27.680766 16755 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:01:27.682930 16755 webserver.cc:460] Webserver started at http://127.4.82.130:41067/ using document root <none> and password file <none>
I20260809 01:01:27.683745 16755 fs_manager.cc:359] Metadata directory not provided
I20260809 01:01:27.683894 16755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:01:27.684247 16755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 01:01:27 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237287, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:01:27.702482 16755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "575250ab7aee4d9d9f74180a01362194"
format_stamp: "Formatted at 2026-08-09 01:01:27 on dist-test-slave-07c2"
server_key: "d2440fd0291ce73dcf03e98038238b83"
server_key_iv: "c440e2eb2741fca170a96944fc995d60"
server_key_version: "kuduclusterkey@0"
I20260809 01:01:27.703598 16755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "575250ab7aee4d9d9f74180a01362194"
format_stamp: "Formatted at 2026-08-09 01:01:27 on dist-test-slave-07c2"
server_key: "d2440fd0291ce73dcf03e98038238b83"
server_key_iv: "c440e2eb2741fca170a96944fc995d60"
server_key_version: "kuduclusterkey@0"
I20260809 01:01:27.710758 16755 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260809 01:01:27.731539 16780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:01:27.733243 16755 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260809 01:01:27.733462 16755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "575250ab7aee4d9d9f74180a01362194"
format_stamp: "Formatted at 2026-08-09 01:01:27 on dist-test-slave-07c2"
server_key: "d2440fd0291ce73dcf03e98038238b83"
server_key_iv: "c440e2eb2741fca170a96944fc995d60"
server_key_version: "kuduclusterkey@0"
I20260809 01:01:27.733812 16755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:01:27.753230 16755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:01:27.758489 16755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:01:27.759038 16755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:01:27.760571 16755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:01:27.763022 16755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:01:27.763171 16755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:01:27.763437 16755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:01:27.763603 16755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 01:01:27.859547 16755 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:42593
I20260809 01:01:27.859603 16902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:42593 every 8 connection(s)
I20260809 01:01:27.861020 16755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:01:27.863916 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 16755
I20260809 01:01:27.864115 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:01:27.864761 4426 external_mini_cluster.cc:1428] Setting key f86e25fa0336cd17e529c3aa1209a1a9
Aug 09 01:01:27 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237287, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:01:27.896704 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:27.871547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:45627 (local address 127.4.82.190:34935)
0809 01:01:27.872286 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:27.872293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:27.872315 (+ 22us) server_negotiation.cc:407] Connection header received
0809 01:01:27.872381 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:27.872387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:27.872480 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:27.874156 (+ 1676us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:01:27.875305 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:27.877302 (+ 1997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:27.881014 (+ 3712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:27.881601 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:27.881703 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:27.888243 (+ 6540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:01:27.888252 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:01:27.888261 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:01:27.888343 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:01:27.892789 (+ 4446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:27.893579 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:27.893584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:27.893586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:27.893796 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:27.894326 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:27.894331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:27.894333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:27.894678 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:01:27.894834 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:27.895866 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0809 01:01:27.896093 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":760,"threads_started":1}
I20260809 01:01:27.897742 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:27.865280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34935 (local address 127.4.82.130:45627)
0809 01:01:27.867032 (+ 1752us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:27.867098 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 01:01:27.869406 (+ 2308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:01:27.872962 (+ 3556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:01:27.872973 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:01:27.873852 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:01:27.875019 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:27.875049 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:27.877538 (+ 2489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:27.877546 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:27.880826 (+ 3280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:27.880843 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:27.881925 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:27.881936 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:27.882501 (+ 565us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:27.883798 (+ 1297us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:01:27.883844 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:01:27.887913 (+ 4069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:01:27.892931 (+ 5018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:01:27.892937 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:01:27.892951 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:01:27.893427 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:01:27.893935 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:01:27.893938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:01:27.893941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:01:27.894192 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:01:27.894820 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:01:27.894828 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:01:27.895216 (+ 388us) client_negotiation.cc:769] Sending connection context
0809 01:01:27.895682 (+ 466us) client_negotiation.cc:240] Negotiation successful
0809 01:01:27.896311 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1397,"thread_start_us":1066,"threads_started":1}
I20260809 01:01:27.899597 16903 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34935
I20260809 01:01:27.899964 16903 heartbeater.cc:464] Registering TS with master...
I20260809 01:01:27.900166 16749 heartbeater.cc:503] Master 127.4.82.190:34935 was elected leader, sending a full tablet report...
I20260809 01:01:27.901166 16903 heartbeater.cc:511] Master 127.4.82.190:34935 requested a full tablet report, sending...
I20260809 01:01:27.902969 16521 ts_manager.cc:194] Registered new tserver with Master: 575250ab7aee4d9d9f74180a01362194 (127.4.82.130:42593)
I20260809 01:01:27.903932 16521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:45627
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 01:01:27.943158 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.82.148:46913/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34935
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:01:28.499465 16909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:01:28.499845 16909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:01:28.500025 16909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:01:28.510023 16909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:01:28.510206 16909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:01:28.510393 16909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:01:28.523885 16909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.82.148:46913/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:01:28.525658 16909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:01:28.527151 16909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:01:28.546954 16914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:01:28.547221 16915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:01:28.547768 16917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:01:28.548877 16909 server_base.cc:1034] running on GCE node
I20260809 01:01:28.549685 16909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:01:28.550905 16909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:01:28.552222 16909 hybrid_clock.cc:648] HybridClock initialized: now 1786237288552097 us; error 48 us; skew 500 ppm
I20260809 01:01:28.552680 16909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:01:28 dist-test-slave-07c2 krb5kdc[16014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237288, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:01:28.557128 16909 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:01:28.559069 16909 webserver.cc:460] Webserver started at http://127.4.82.131:42601/ using document root <none> and password file <none>
I20260809 01:01:28.560021 16909 fs_manager.cc:359] Metadata directory not provided
I20260809 01:01:28.560261 16909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:01:28.560709 16909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 01:01:28 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237288, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:01:28.578158 16909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "66fdb999ec9148d6b9fb844ebba05bcb"
format_stamp: "Formatted at 2026-08-09 01:01:28 on dist-test-slave-07c2"
server_key: "00cdbe9b7dcad31766339e64301b3818"
server_key_iv: "6c6f59afeaec70b315a55a55c472512a"
server_key_version: "kuduclusterkey@0"
I20260809 01:01:28.579373 16909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "66fdb999ec9148d6b9fb844ebba05bcb"
format_stamp: "Formatted at 2026-08-09 01:01:28 on dist-test-slave-07c2"
server_key: "00cdbe9b7dcad31766339e64301b3818"
server_key_iv: "6c6f59afeaec70b315a55a55c472512a"
server_key_version: "kuduclusterkey@0"
I20260809 01:01:28.586335 16909 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260809 01:01:28.605101 16930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:01:28.606827 16909 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260809 01:01:28.607064 16909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "66fdb999ec9148d6b9fb844ebba05bcb"
format_stamp: "Formatted at 2026-08-09 01:01:28 on dist-test-slave-07c2"
server_key: "00cdbe9b7dcad31766339e64301b3818"
server_key_iv: "6c6f59afeaec70b315a55a55c472512a"
server_key_version: "kuduclusterkey@0"
I20260809 01:01:28.607411 16909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:01:28.631673 16909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:01:28.637418 16909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:01:28.638023 16909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:01:28.639794 16909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:01:28.642035 16909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:01:28.642237 16909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:01:28.642479 16909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:01:28.642622 16909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:01:28.727195 16909 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:42093
I20260809 01:01:28.727465 17051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:42093 every 8 connection(s)
I20260809 01:01:28.728808 16909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:01:28.735553 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 16909
I20260809 01:01:28.735765 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:01:28.736626 4426 external_mini_cluster.cc:1428] Setting key 2ae794b157e0f93d4c19b44e1a311232
Aug 09 01:01:28 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237288, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:01:28.759092 17054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.732210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:53959 (local address 127.4.82.190:34935)
0809 01:01:28.733158 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:28.733163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:28.734483 (+ 1320us) server_negotiation.cc:407] Connection header received
0809 01:01:28.739812 (+ 5329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:28.739817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:28.739991 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:28.740104 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:01:28.742088 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.743953 (+ 1865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.745963 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.747091 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.747187 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:28.751790 (+ 4603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:01:28.751798 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:01:28.751804 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:01:28.751898 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:01:28.755927 (+ 4029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:28.756720 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:28.756724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:28.756726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:28.756902 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:28.757430 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:28.757433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:28.757435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:28.757773 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:01:28.757919 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:28.758523 (+ 604us) server_negotiation.cc:299] Negotiation successful
0809 01:01:28.758728 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":807,"thread_start_us":636,"threads_started":1}
I20260809 01:01:28.760232 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.732858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34935 (local address 127.4.82.131:53959)
0809 01:01:28.734279 (+ 1421us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:28.734316 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 01:01:28.739402 (+ 5086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:01:28.740495 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:01:28.740502 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:01:28.741171 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:01:28.741884 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.741902 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.744107 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.744111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:28.745745 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.745757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.747424 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.747431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:28.747716 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:28.748371 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:01:28.748394 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:01:28.751427 (+ 3033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:01:28.756046 (+ 4619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:01:28.756052 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:01:28.756065 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:01:28.756608 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:01:28.756998 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:01:28.757001 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:01:28.757003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:01:28.757333 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:01:28.757881 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:01:28.757887 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:01:28.758215 (+ 328us) client_negotiation.cc:769] Sending connection context
0809 01:01:28.758440 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 01:01:28.758883 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1166,"thread_start_us":854,"threads_started":1}
I20260809 01:01:28.761833 17052 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34935
I20260809 01:01:28.762145 17052 heartbeater.cc:464] Registering TS with master...
I20260809 01:01:28.762888 17052 heartbeater.cc:511] Master 127.4.82.190:34935 requested a full tablet report, sending...
I20260809 01:01:28.764927 16521 ts_manager.cc:194] Registered new tserver with Master: 66fdb999ec9148d6b9fb844ebba05bcb (127.4.82.131:42093)
I20260809 01:01:28.765743 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:01:28.766175 16521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:53959
I20260809 01:01:28.790452 17054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.772853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52934 (local address 127.4.82.190:34935)
0809 01:01:28.773036 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:28.773040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:28.773916 (+ 876us) server_negotiation.cc:407] Connection header received
0809 01:01:28.774145 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:28.774151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:28.774317 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:28.774440 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:01:28.776389 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.777346 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.779260 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.780129 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.780273 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:28.783672 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:01:28.783677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:01:28.783680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:01:28.783709 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:01:28.787104 (+ 3395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:28.787853 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:28.787860 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:28.787863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:28.788051 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:28.788532 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:28.788538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:28.788554 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:28.788898 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:01:28.789289 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:28.789622 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 01:01:28.789921 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 01:01:28.794837 16521 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:01:28.805377 16521 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:01:28.855398 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.824469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33410 (local address 127.4.82.130:42593)
0809 01:01:28.838398 (+ 13929us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:28.838407 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:28.840632 (+ 2225us) server_negotiation.cc:407] Connection header received
0809 01:01:28.840895 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:28.840907 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:28.841311 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:28.841553 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:01:28.843512 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.845900 (+ 2388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.850942 (+ 5042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.853408 (+ 2466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.853606 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:28.853794 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:28.854550 (+ 756us) server_negotiation.cc:299] Negotiation successful
0809 01:01:28.854756 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13421,"thread_start_us":13543,"threads_started":1}
I20260809 01:01:28.855397 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.823702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:42593 (local address 127.0.0.1:33410)
0809 01:01:28.840434 (+ 16732us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:28.840460 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 01:01:28.840699 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:01:28.842034 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:01:28.842042 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:01:28.842126 (+ 84us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:01:28.843154 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.843168 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.846151 (+ 2983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.846156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:28.850665 (+ 4509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.850690 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.854082 (+ 3392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.854089 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:28.854287 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:28.854354 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 01:01:28.854494 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 01:01:28.854718 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16510,"thread_start_us":28008,"threads_started":1}
I20260809 01:01:28.860759 16831 tablet_service.cc:1467] Processing CreateTablet for tablet 6c065b587a274ae3b1eb0245f793c049 (DEFAULT_TABLE table=test-table [id=a3284624090c4b1ba5716b1605793093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:01:28.863022 16831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c065b587a274ae3b1eb0245f793c049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:01:28.864825 17067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.823293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:42093 (local address 127.0.0.1:56784)
0809 01:01:28.830905 (+ 7612us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:28.830987 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0809 01:01:28.831235 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:01:28.851804 (+ 20569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:01:28.851810 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:01:28.851854 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:01:28.852929 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.852940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.857919 (+ 4979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.857943 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:28.861322 (+ 3379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.861337 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.863616 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.863623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:28.863871 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:28.863907 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 01:01:28.863978 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 01:01:28.864120 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7370,"mutex_wait_us":4,"thread_start_us":7064,"threads_started":1}
I20260809 01:01:28.866500 17068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.823179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56784 (local address 127.4.82.131:42093)
0809 01:01:28.850981 (+ 27802us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:28.850987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:28.851030 (+ 43us) server_negotiation.cc:407] Connection header received
0809 01:01:28.851107 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:28.851112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:28.851419 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:28.853149 (+ 1730us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:01:28.853958 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.857759 (+ 3801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.861580 (+ 3821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.863344 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.865087 (+ 1743us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:28.865407 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:28.865478 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 01:01:28.865748 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27471,"thread_start_us":27003,"threads_started":1}
I20260809 01:01:28.867370 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.837764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:38193 (local address 127.0.0.1:38080)
0809 01:01:28.852236 (+ 14472us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:28.852267 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 01:01:28.852466 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:01:28.853144 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:01:28.853149 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:01:28.853176 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:01:28.854669 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.854685 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.857973 (+ 3288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.857978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:28.862982 (+ 5004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.863006 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.866114 (+ 3108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.866120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:28.866380 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:28.866414 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 01:01:28.866506 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 01:01:28.866658 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14223,"mutex_wait_us":1,"thread_start_us":2167,"threads_started":1}
I20260809 01:01:28.868475 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.831779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38080 (local address 127.4.82.129:38193)
0809 01:01:28.849639 (+ 17860us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:28.849648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:28.852407 (+ 2759us) server_negotiation.cc:407] Connection header received
0809 01:01:28.852693 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:28.852701 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:28.852910 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:28.853184 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:01:28.855040 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.857665 (+ 2625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.863307 (+ 5642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.865273 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.865459 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:28.865811 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:28.867679 (+ 1868us) server_negotiation.cc:299] Negotiation successful
0809 01:01:28.867861 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17593,"thread_start_us":17865,"threads_started":1}
I20260809 01:01:28.871069 16984 tablet_service.cc:1467] Processing CreateTablet for tablet 6c065b587a274ae3b1eb0245f793c049 (DEFAULT_TABLE table=test-table [id=a3284624090c4b1ba5716b1605793093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:01:28.873260 16984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c065b587a274ae3b1eb0245f793c049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:01:28.872941 16678 tablet_service.cc:1467] Processing CreateTablet for tablet 6c065b587a274ae3b1eb0245f793c049 (DEFAULT_TABLE table=test-table [id=a3284624090c4b1ba5716b1605793093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:01:28.874454 16678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c065b587a274ae3b1eb0245f793c049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:01:28.892380 17074 tablet_bootstrap.cc:492] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194: Bootstrap starting.
I20260809 01:01:28.898272 17074 tablet_bootstrap.cc:654] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194: Neither blocks nor log segments found. Creating new log.
I20260809 01:01:28.901360 17076 tablet_bootstrap.cc:492] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb: Bootstrap starting.
I20260809 01:01:28.902781 17074 log.cc:824] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194: Log is configured to *not* fsync() on all Append() calls
I20260809 01:01:28.906910 16903 heartbeater.cc:503] Master 127.4.82.190:34935 was elected leader, sending a full tablet report...
I20260809 01:01:28.910259 17075 tablet_bootstrap.cc:492] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d: Bootstrap starting.
I20260809 01:01:28.912308 17074 tablet_bootstrap.cc:492] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194: No bootstrap required, opened a new log
I20260809 01:01:28.912768 17074 ts_tablet_manager.cc:1397] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.017s
I20260809 01:01:28.916705 17076 tablet_bootstrap.cc:654] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb: Neither blocks nor log segments found. Creating new log.
I20260809 01:01:28.919286 17076 log.cc:824] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb: Log is configured to *not* fsync() on all Append() calls
I20260809 01:01:28.919855 17075 tablet_bootstrap.cc:654] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d: Neither blocks nor log segments found. Creating new log.
I20260809 01:01:28.922499 17075 log.cc:824] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d: Log is configured to *not* fsync() on all Append() calls
I20260809 01:01:28.924655 17076 tablet_bootstrap.cc:492] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb: No bootstrap required, opened a new log
I20260809 01:01:28.925108 17076 ts_tablet_manager.cc:1397] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb: Time spent bootstrapping tablet: real 0.024s user 0.013s sys 0.000s
I20260809 01:01:28.926731 17075 tablet_bootstrap.cc:492] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d: No bootstrap required, opened a new log
I20260809 01:01:28.927136 17075 ts_tablet_manager.cc:1397] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.004s
I20260809 01:01:28.929687 17074 raft_consensus.cc:348] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42093 } } peers { permanent_uuid: "575250ab7aee4d9d9f74180a01362194" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42593 } } peers { permanent_uuid: "96ae17813b7f403e8397237a4e24233d" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 38193 } }
I20260809 01:01:28.930398 17074 raft_consensus.cc:374] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:01:28.930503 17074 raft_consensus.cc:729] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575250ab7aee4d9d9f74180a01362194, State: Initialized, Role: FOLLOWER
I20260809 01:01:28.931257 17074 consensus_queue.cc:260] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42093 } } peers { permanent_uuid: "575250ab7aee4d9d9f74180a01362194" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42593 } } peers { permanent_uuid: "96ae17813b7f403e8397237a4e24233d" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 38193 } }
I20260809 01:01:28.933938 17074 ts_tablet_manager.cc:1428] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194: Time spent starting tablet: real 0.021s user 0.007s sys 0.013s
I20260809 01:01:28.940543 17076 raft_consensus.cc:348] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42093 } } peers { permanent_uuid: "575250ab7aee4d9d9f74180a01362194" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42593 } } peers { permanent_uuid: "96ae17813b7f403e8397237a4e24233d" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 38193 } }
I20260809 01:01:28.941076 17076 raft_consensus.cc:374] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:01:28.940790 17075 raft_consensus.cc:348] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42093 } } peers { permanent_uuid: "575250ab7aee4d9d9f74180a01362194" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42593 } } peers { permanent_uuid: "96ae17813b7f403e8397237a4e24233d" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 38193 } }
I20260809 01:01:28.941237 17076 raft_consensus.cc:729] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66fdb999ec9148d6b9fb844ebba05bcb, State: Initialized, Role: FOLLOWER
I20260809 01:01:28.941356 17075 raft_consensus.cc:374] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:01:28.941571 17075 raft_consensus.cc:729] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96ae17813b7f403e8397237a4e24233d, State: Initialized, Role: FOLLOWER
I20260809 01:01:28.941969 17076 consensus_queue.cc:260] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42093 } } peers { permanent_uuid: "575250ab7aee4d9d9f74180a01362194" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42593 } } peers { permanent_uuid: "96ae17813b7f403e8397237a4e24233d" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 38193 } }
I20260809 01:01:28.942216 17075 consensus_queue.cc:260] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42093 } } peers { permanent_uuid: "575250ab7aee4d9d9f74180a01362194" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42593 } } peers { permanent_uuid: "96ae17813b7f403e8397237a4e24233d" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 38193 } }
I20260809 01:01:28.944017 17052 heartbeater.cc:503] Master 127.4.82.190:34935 was elected leader, sending a full tablet report...
I20260809 01:01:28.944571 17076 ts_tablet_manager.cc:1428] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb: Time spent starting tablet: real 0.019s user 0.005s sys 0.014s
I20260809 01:01:28.945560 17075 ts_tablet_manager.cc:1428] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
I20260809 01:01:28.972656 17082 raft_consensus.cc:482] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:01:28.972954 17082 raft_consensus.cc:504] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42093 } } peers { permanent_uuid: "575250ab7aee4d9d9f74180a01362194" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42593 } } peers { permanent_uuid: "96ae17813b7f403e8397237a4e24233d" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 38193 } }
I20260809 01:01:28.975411 17082 leader_election.cc:288] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66fdb999ec9148d6b9fb844ebba05bcb (127.4.82.131:42093), 575250ab7aee4d9d9f74180a01362194 (127.4.82.130:42593)
W20260809 01:01:28.981705 17053 tablet.cc:2369] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:01:28.986721 17068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.975697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:40865 (local address 127.4.82.131:42093)
0809 01:01:28.976127 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:28.976134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:28.978181 (+ 2047us) server_negotiation.cc:407] Connection header received
0809 01:01:28.978417 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:28.978422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:28.978489 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:28.978620 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:01:28.979539 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.980655 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.984402 (+ 3747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.985421 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.985658 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:28.985774 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:28.986143 (+ 369us) server_negotiation.cc:299] Negotiation successful
0809 01:01:28.986268 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260809 01:01:28.987090 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.976288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:42093 (local address 127.4.82.129:40865)
0809 01:01:28.978033 (+ 1745us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:28.978050 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:01:28.978274 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:01:28.978651 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:01:28.978657 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:01:28.978683 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:01:28.979392 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.979404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.980785 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.980793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:28.984251 (+ 3458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.984263 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.985789 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.985800 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:28.985969 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:28.985992 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 01:01:28.986117 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 01:01:28.986249 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1499,"mutex_wait_us":1,"thread_start_us":1128,"threads_started":1}
I20260809 01:01:28.987309 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.976112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:50427 (local address 127.4.82.130:42593)
0809 01:01:28.976350 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:28.976355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:28.978551 (+ 2196us) server_negotiation.cc:407] Connection header received
0809 01:01:28.978827 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:28.978831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:28.978888 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:28.978985 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:01:28.980201 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.981261 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.984586 (+ 3325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.985841 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.986083 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:28.986192 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:28.986605 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 01:01:28.986733 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 01:01:28.987516 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.976541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:42593 (local address 127.4.82.129:50427)
0809 01:01:28.978414 (+ 1873us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:28.978447 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 01:01:28.978635 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:01:28.979002 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:01:28.979006 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:01:28.979029 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:01:28.980005 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.980016 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.981447 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.981451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:28.984390 (+ 2939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.984402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.986243 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.986253 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:28.986432 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:28.986453 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 01:01:28.986562 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 01:01:28.986681 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1729,"mutex_wait_us":4,"thread_start_us":1496,"threads_started":1}
I20260809 01:01:28.988426 17006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c065b587a274ae3b1eb0245f793c049" candidate_uuid: "96ae17813b7f403e8397237a4e24233d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" is_pre_election: true
I20260809 01:01:28.989010 16855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c065b587a274ae3b1eb0245f793c049" candidate_uuid: "96ae17813b7f403e8397237a4e24233d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575250ab7aee4d9d9f74180a01362194" is_pre_election: true
I20260809 01:01:28.989351 17006 raft_consensus.cc:2454] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96ae17813b7f403e8397237a4e24233d in term 0.
I20260809 01:01:28.989907 17081 raft_consensus.cc:482] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:01:28.989813 16855 raft_consensus.cc:2454] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96ae17813b7f403e8397237a4e24233d in term 0.
I20260809 01:01:28.990909 16627 leader_election.cc:302] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66fdb999ec9148d6b9fb844ebba05bcb, 96ae17813b7f403e8397237a4e24233d; no voters:
I20260809 01:01:28.990528 17081 raft_consensus.cc:504] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42093 } } peers { permanent_uuid: "575250ab7aee4d9d9f74180a01362194" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42593 } } peers { permanent_uuid: "96ae17813b7f403e8397237a4e24233d" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 38193 } }
I20260809 01:01:28.991725 17082 raft_consensus.cc:2781] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:01:28.992017 17082 raft_consensus.cc:482] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:01:28.992240 17082 raft_consensus.cc:3037] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:01:28.993168 17081 leader_election.cc:288] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 575250ab7aee4d9d9f74180a01362194 (127.4.82.130:42593), 96ae17813b7f403e8397237a4e24233d (127.4.82.129:38193)
I20260809 01:01:28.997287 17082 raft_consensus.cc:504] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42093 } } peers { permanent_uuid: "575250ab7aee4d9d9f74180a01362194" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42593 } } peers { permanent_uuid: "96ae17813b7f403e8397237a4e24233d" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 38193 } }
I20260809 01:01:28.999369 16855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c065b587a274ae3b1eb0245f793c049" candidate_uuid: "96ae17813b7f403e8397237a4e24233d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575250ab7aee4d9d9f74180a01362194"
I20260809 01:01:28.999372 17006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c065b587a274ae3b1eb0245f793c049" candidate_uuid: "96ae17813b7f403e8397237a4e24233d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66fdb999ec9148d6b9fb844ebba05bcb"
I20260809 01:01:28.999644 17006 raft_consensus.cc:3037] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:01:28.999660 16855 raft_consensus.cc:3037] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:01:29.000811 17082 leader_election.cc:288] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [CANDIDATE]: Term 1 election: Requested vote from peers 66fdb999ec9148d6b9fb844ebba05bcb (127.4.82.131:42093), 575250ab7aee4d9d9f74180a01362194 (127.4.82.130:42593)
I20260809 01:01:29.004732 17055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.994387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:42593 (local address 127.4.82.131:54409)
0809 01:01:28.994963 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:28.994978 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 01:01:28.995123 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:01:28.995493 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:01:28.995497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:01:28.995524 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:01:28.996107 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.996117 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.997693 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:28.997699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:29.000752 (+ 3053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:29.000770 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.003445 (+ 2675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.003461 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:29.003904 (+ 443us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:29.003969 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 01:01:29.004117 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 01:01:29.004324 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":5}
I20260809 01:01:29.005514 16855 raft_consensus.cc:2454] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96ae17813b7f403e8397237a4e24233d in term 1.
I20260809 01:01:29.005617 17006 raft_consensus.cc:2454] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96ae17813b7f403e8397237a4e24233d in term 1.
I20260809 01:01:29.005975 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.993917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:54409 (local address 127.4.82.130:42593)
0809 01:01:28.994110 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:28.994118 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:28.995059 (+ 941us) server_negotiation.cc:407] Connection header received
0809 01:01:28.995252 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:28.995260 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:28.995387 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:28.995477 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:01:28.996252 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:28.997531 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.001034 (+ 3503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.003181 (+ 2147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.005032 (+ 1851us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:29.005114 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:29.005165 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 01:01:29.005366 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 01:01:29.006238 16630 leader_election.cc:302] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575250ab7aee4d9d9f74180a01362194, 96ae17813b7f403e8397237a4e24233d; no voters:
I20260809 01:01:29.006820 17082 raft_consensus.cc:2781] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:01:29.007006 16855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c065b587a274ae3b1eb0245f793c049" candidate_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "575250ab7aee4d9d9f74180a01362194" is_pre_election: true
I20260809 01:01:29.007376 16855 raft_consensus.cc:2379] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 66fdb999ec9148d6b9fb844ebba05bcb in current term 1: Already voted for candidate 96ae17813b7f403e8397237a4e24233d in this term.
I20260809 01:01:29.008153 17082 raft_consensus.cc:686] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [term 1 LEADER]: Becoming Leader. State: Replica: 96ae17813b7f403e8397237a4e24233d, State: Running, Role: LEADER
I20260809 01:01:29.009300 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.993663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:53249 (local address 127.4.82.129:38193)
0809 01:01:28.993901 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:28.993916 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:28.996239 (+ 2323us) server_negotiation.cc:407] Connection header received
0809 01:01:28.996558 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:28.996572 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:28.996645 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:28.996759 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:01:28.998594 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.000227 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.005419 (+ 5192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.007509 (+ 2090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.007878 (+ 369us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:29.007993 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:29.008878 (+ 885us) server_negotiation.cc:299] Negotiation successful
0809 01:01:29.009025 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 01:01:29.009615 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:28.994428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:38193 (local address 127.4.82.131:53249)
0809 01:01:28.996140 (+ 1712us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:28.996159 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 01:01:28.996383 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:01:28.996799 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:01:28.996807 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:01:28.996838 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:01:28.998242 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:28.998257 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.000629 (+ 2372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.000634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:29.005231 (+ 4597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:01:29.005248 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.008321 (+ 3073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.008327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:01:29.008641 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:29.008680 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 01:01:29.008827 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 01:01:29.008971 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1517,"thread_start_us":911,"threads_started":1}
I20260809 01:01:29.009399 17082 consensus_queue.cc:237] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42093 } } peers { permanent_uuid: "575250ab7aee4d9d9f74180a01362194" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42593 } } peers { permanent_uuid: "96ae17813b7f403e8397237a4e24233d" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 38193 } }
I20260809 01:01:29.011011 16700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c065b587a274ae3b1eb0245f793c049" candidate_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96ae17813b7f403e8397237a4e24233d" is_pre_election: true
I20260809 01:01:29.012691 16931 leader_election.cc:302] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66fdb999ec9148d6b9fb844ebba05bcb; no voters: 575250ab7aee4d9d9f74180a01362194, 96ae17813b7f403e8397237a4e24233d
I20260809 01:01:29.013299 17081 raft_consensus.cc:2726] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 01:01:29.015609 16521 catalog_manager.cc:5495] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d reported cstate change: term changed from 0 to 1, leader changed from <none> to 96ae17813b7f403e8397237a4e24233d (127.4.82.129). New cstate: current_term: 1 leader_uuid: "96ae17813b7f403e8397237a4e24233d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "575250ab7aee4d9d9f74180a01362194" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96ae17813b7f403e8397237a4e24233d" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 38193 } health_report { overall_health: HEALTHY } } }
I20260809 01:01:29.034574 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:29.027497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38094 (local address 127.4.82.129:38193)
0809 01:01:29.027731 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:29.027737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:29.027840 (+ 103us) server_negotiation.cc:407] Connection header received
0809 01:01:29.028022 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:29.028025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:29.028167 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:29.028284 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:01:29.028836 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.029869 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.031974 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.032711 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.032840 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:29.033340 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:01:29.033797 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:01:29.033900 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:29.034000 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 01:01:29.034060 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 01:01:29.043968 17006 raft_consensus.cc:1261] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [term 1 FOLLOWER]: Refusing update from remote peer 96ae17813b7f403e8397237a4e24233d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:01:29.044077 16855 raft_consensus.cc:1261] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194 [term 1 FOLLOWER]: Refusing update from remote peer 96ae17813b7f403e8397237a4e24233d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:01:29.044865 17086 consensus_queue.cc:1035] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [LEADER]: Connected to new peer: Peer: permanent_uuid: "66fdb999ec9148d6b9fb844ebba05bcb" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:01:29.045377 17082 consensus_queue.cc:1035] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [LEADER]: Connected to new peer: Peer: permanent_uuid: "575250ab7aee4d9d9f74180a01362194" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 42593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:01:29.060037 17089 mvcc.cc:205] Tried to move back new op lower bound from 7316427935914418176 to 7316427935787257856. Current Snapshot: MvccSnapshot[applied={T|T < 7316427935914418176}]
I20260809 01:01:29.066569 17091 mvcc.cc:205] Tried to move back new op lower bound from 7316427935914418176 to 7316427935787257856. Current Snapshot: MvccSnapshot[applied={T|T < 7316427935914418176 or (T in {7316427935914418176})}]
I20260809 01:01:29.080315 16520 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:01:29.080979 16520 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:01:29.086723 16520 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc7a9977ab3d48a19ef39e2b61378496: Sending DeleteTablet for 3 replicas of tablet 6c065b587a274ae3b1eb0245f793c049
I20260809 01:01:29.088100 16984 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c065b587a274ae3b1eb0245f793c049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:01:29 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:56784
I20260809 01:01:29.088253 16831 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c065b587a274ae3b1eb0245f793c049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:01:29 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:33410
I20260809 01:01:29.089455 16678 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c065b587a274ae3b1eb0245f793c049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:01:29 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:38080
I20260809 01:01:29.092558 17100 tablet_replica.cc:330] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194: stopping tablet replica
I20260809 01:01:29.092585 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 16606
I20260809 01:01:29.093096 17100 raft_consensus.cc:2229] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:01:29.093724 17100 raft_consensus.cc:2258] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:01:29.094637 17099 tablet_replica.cc:330] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb: stopping tablet replica
I20260809 01:01:29.095085 17099 raft_consensus.cc:2229] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:01:29.095662 17099 raft_consensus.cc:2258] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:01:29.098351 17099 ts_tablet_manager.cc:1905] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:01:29.099776 17101 tablet_replica.cc:330] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d: stopping tablet replica
I20260809 01:01:29.100874 17100 ts_tablet_manager.cc:1905] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:01:29.100396 17101 raft_consensus.cc:2229] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:01:29.105042 17101 raft_consensus.cc:2258] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 01:01:29 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 01:01:29.110713 17100 ts_tablet_manager.cc:1918] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:01:29.111032 17100 log.cc:1196] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/6c065b587a274ae3b1eb0245f793c049
I20260809 01:01:29.111719 17101 ts_tablet_manager.cc:1905] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:01:29.112056 17100 ts_tablet_manager.cc:1939] T 6c065b587a274ae3b1eb0245f793c049 P 575250ab7aee4d9d9f74180a01362194: Deleting consensus metadata
I20260809 01:01:29.113809 17099 ts_tablet_manager.cc:1918] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:01:29.114014 17099 log.cc:1196] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/6c065b587a274ae3b1eb0245f793c049
I20260809 01:01:29.114586 17099 ts_tablet_manager.cc:1939] T 6c065b587a274ae3b1eb0245f793c049 P 66fdb999ec9148d6b9fb844ebba05bcb: Deleting consensus metadata
I20260809 01:01:29.114683 16508 catalog_manager.cc:4841] TS 575250ab7aee4d9d9f74180a01362194 (127.4.82.130:42593): tablet 6c065b587a274ae3b1eb0245f793c049 (table test-table [id=a3284624090c4b1ba5716b1605793093]) successfully deleted
I20260809 01:01:29.116921 16505 catalog_manager.cc:4841] TS 66fdb999ec9148d6b9fb844ebba05bcb (127.4.82.131:42093): tablet 6c065b587a274ae3b1eb0245f793c049 (table test-table [id=a3284624090c4b1ba5716b1605793093]) successfully deleted
I20260809 01:01:29.124893 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:29.094056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38110 (local address 127.4.82.129:38193)
0809 01:01:29.094275 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:29.094283 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:29.094872 (+ 589us) server_negotiation.cc:407] Connection header received
0809 01:01:29.095250 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:29.095255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:29.095471 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:29.095562 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:01:29.099228 (+ 3666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.100501 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.101936 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.103057 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.103187 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:29.114735 (+ 11548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:01:29.114776 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:01:29.114803 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:01:29.114867 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:01:29.120064 (+ 5197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:29.121433 (+ 1369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:29.121444 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:29.121448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:29.121709 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:29.122539 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:29.122544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:29.122548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:29.123204 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:01:29.123692 (+ 488us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:29.124179 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 01:01:29.124481 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 01:01:29.126078 17101 ts_tablet_manager.cc:1918] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:01:29.126153 16744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:01:29.126410 17101 log.cc:1196] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/6c065b587a274ae3b1eb0245f793c049
I20260809 01:01:29.226105 17101 ts_tablet_manager.cc:1939] T 6c065b587a274ae3b1eb0245f793c049 P 96ae17813b7f403e8397237a4e24233d: Deleting consensus metadata
I20260809 01:01:29.227463 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 16606
W20260809 01:01:29.248917 16505 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.82.129:38193 (error 108)
I20260809 01:01:29.249264 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 16755
I20260809 01:01:29.251065 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:29.250569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:38193 (local address 127.0.0.1:38122)
0809 01:01:29.250767 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:29.250875 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:38193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 01:01:29.251400 16505 catalog_manager.cc:4584] TS 96ae17813b7f403e8397237a4e24233d (127.4.82.129:38193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c065b587a274ae3b1eb0245f793c049: Network error: Client connection negotiation failed: client connection to 127.4.82.129:38193: connect: Connection refused (error 111)
Aug 09 01:01:29 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 01:01:29.265355 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:29.249983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33414 (local address 127.4.82.130:42593)
0809 01:01:29.250160 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:29.250168 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:29.250345 (+ 177us) server_negotiation.cc:407] Connection header received
0809 01:01:29.250549 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:29.250554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:29.250642 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:29.250766 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:01:29.252598 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.253324 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.254861 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.255358 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.255501 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:29.260158 (+ 4657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:01:29.260170 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:01:29.260184 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:01:29.260215 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:01:29.262791 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:29.263495 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:29.263502 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:29.263504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:29.263636 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:29.263990 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:29.263993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:29.263994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:29.264307 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:01:29.264532 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:29.264827 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 01:01:29.265045 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 01:01:29.266074 16898 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:01:29.305141 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:29.304557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:38193 (local address 127.0.0.1:38130)
0809 01:01:29.304781 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:29.304928 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:38193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 01:01:29.306180 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:29.305810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:38193 (local address 127.0.0.1:38146)
0809 01:01:29.305959 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:29.306039 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:38193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 01:01:29.357455 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 16755
I20260809 01:01:29.366873 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:29.366444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:38193 (local address 127.0.0.1:38154)
0809 01:01:29.366622 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:29.366724 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:38193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 01:01:29.367831 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:29.367501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:38193 (local address 127.0.0.1:38162)
0809 01:01:29.367647 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:29.367724 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:38193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 01:01:29.376900 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 16909
Aug 09 01:01:29 dist-test-slave-07c2 krb5kdc[16014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 01:01:29.392393 17068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:29.377585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56796 (local address 127.4.82.131:42093)
0809 01:01:29.377764 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 01:01:29.377768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:01:29.377786 (+ 18us) server_negotiation.cc:407] Connection header received
0809 01:01:29.377920 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:01:29.377924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:01:29.377987 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:01:29.378071 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:01:29.379389 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.380124 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.381478 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:01:29.381863 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:01:29.381956 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:01:29.386335 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:01:29.386349 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:01:29.386365 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:01:29.386401 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:01:29.389134 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:29.389766 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:29.389773 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:29.389775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:29.389895 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:01:29.390397 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:01:29.390405 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:01:29.390410 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:01:29.390940 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:01:29.391509 (+ 569us) server_negotiation.cc:1035] Waiting for connection context
0809 01:01:29.391672 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 01:01:29.391928 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 01:01:29.393132 17047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:01:29.448378 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:29.447806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:38193 (local address 127.0.0.1:38168)
0809 01:01:29.448042 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:29.448167 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:38193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 01:01:29.449496 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:01:29.449064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:38193 (local address 127.0.0.1:38180)
0809 01:01:29.449270 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 01:01:29.449356 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:38193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 01:01:29.489198 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 16909
I20260809 01:01:29.507961 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 16488
I20260809 01:01:29.509195 16555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:01:29.645262 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 16488
I20260809 01:01:29.669164 4426 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 01:01:30.013861 4426 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 01:01:30.014137 4426 mini_ranger.cc:66] Stopping Ranger...
I20260809 01:01:30.358880 4426 mini_ranger.cc:68] Stopped Ranger
2026-08-09 01:01:30.359 UTC [16047] LOG: received smart shutdown request
2026-08-09 01:01:30.362 UTC [16047] LOG: background worker "logical replication launcher" (PID 16056) exited with exit code 1
2026-08-09 01:01:30.363 UTC [16051] LOG: shutting down
2026-08-09 01:01:30.522 UTC [16047] LOG: database system is shut down
2026-08-09T01:01:30Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50270 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:01:30 dist-test-slave-07c2 krb5kdc[17116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:01:30 dist-test-slave-07c2 krb5kdc[17116](info): set up 2 sockets
Aug 09 01:01:30 dist-test-slave-07c2 krb5kdc[17116](info): commencing operation
krb5kdc: starting...
W20260809 01:01:31.849301 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:01:32 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:01:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:01:32Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.82.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.82.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/rangeradmin_127.4.82.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/rangeradmin_127.4.82.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.82.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.82.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/rangerlookup_127.4.82.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/rangerlookup_127.4.82.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.82.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.82.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/HTTP_127.4.82.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/HTTP_127.4.82.148@KRBTEST.COM.keytab.
I20260809 01:01:32.204044 4426 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskua6Q0e/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/postgres -l logfile start
2026-08-09 01:01:34.580 UTC [17148] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 01:01:34.580 UTC [17148] LOG: listening on IPv4 address "127.4.82.148", port 42071
2026-08-09 01:01:34.584 UTC [17148] LOG: listening on Unix socket "/tmp/.s.PGSQL.42071"
2026-08-09 01:01:34.607 UTC [17151] LOG: database system was shut down at 2026-08-09 01:01:33 UTC
2026-08-09 01:01:34.610 UTC [17148] LOG: database system is ready to accept connections
I20260809 01:01:35.622622 4426 mini_postgres.cc:96] Postgres bound to 42071
2026-08-09 01:01:35.644 UTC [17163] FATAL: database "slave" does not exist
127.4.82.148:42071 - accepting connections
I20260809 01:01:35.644961 4426 mini_ranger.cc:161] Starting Ranger...
I20260809 01:01:35.673687 4426 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 01:01:36.094648 4426 mini_ranger.cc:87] Created ranger Postgres database
I20260809 01:01:36.094787 4426 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-admin
2026-08-09 01:01:36,374 [I] DB FLAVOR :POSTGRES
2026-08-09 01:01:36,375 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 01:01:36,375 [I] Checking connection..
2026-08-09 01:01:36,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 01:01:36,645 [I] Checking connection passed.
2026-08-09 01:01:36,645 [I] --------- Verifying version history table ---------
2026-08-09 01:01:36,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 01:01:36,901 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 01:01:36,902 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 01:01:37,161 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 01:01:37,162 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 01:01:37,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 01:01:37.389 UTC [17232] WARNING: there is no transaction in progress
2026-08-09 01:01:37,397 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 01:01:37,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 01:01:37,655 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 01:01:37,655 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 01:01:37,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 01:01:37,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 01:01:38,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-07c2.c.gcp-upstream.internal','N') ;"
2026-08-09 01:01:38,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 01:01:38,619 [I] Table x_portal_user does not exist in database ranger
2026-08-09 01:01:38,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 01:01:38,881 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 01:01:38,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 01:01:39,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 01:01:39,375 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 01:01:39,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 01:01:40.260 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.275 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.290 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.303 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.318 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.334 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.393 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.399 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.411 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.421 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.431 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.444 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.453 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.459 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.464 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:40.470 UTC [17348] WARNING: there is no transaction in progress
2026-08-09 01:01:41,051 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 01:01:41,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 01:01:41,303 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 01:01:41,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 01:01:41,546 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 01:01:41,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 01:01:41,802 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 01:01:41,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 01:01:42,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 01:01:42,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-07c2.c.gcp-upstream.internal';"
2026-08-09 01:01:42,530 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 01:01:42,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 01:01:42,791 [I] DB_PATCHES have already been applied
I20260809 01:01:42.797848 4426 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-admin:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskua6Q0e/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 01:01:42.797940 4426 mini_ranger.cc:193] Using host: 127.4.82.148
I20260809 01:01:42.817368 4426 mini_ranger.cc:227] Ranger admin URL: http://127.4.82.148:40071
Aug 09, 2026 1:01:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 1:01:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 1:01:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 1:01:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 1:01:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 1:01:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 1:01:43 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 01:01:44.017271 4426 mini_ranger.cc:160] Time spent starting Ranger: real 8.372s user 0.003s sys 0.048s
Aug 09, 2026 1:02:03 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 1:02:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 09, 2026 1:02:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 1:02:03 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 1:02:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 1:02:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 1:02:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 1:02:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 1:02:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 1:02:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 1:02:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 1:02:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 1:02:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 1:02:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 1:02:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 1:02:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 1:02:03 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 1:02:04 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 01:02:04.922012 4426 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.82.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.82.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/rangerkms_127.4.82.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.82.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/rangerkms_127.4.82.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.82.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.82.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.82.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/HTTP_127.4.82.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.82.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/HTTP_127.4.82.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 01:02:05 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237325, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 01:02:05.107539 4426 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 01:02:05.131125 4426 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 01:02:05.900031 4426 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 01:02:05.900166 4426 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-kms
I20260809 01:02:05.900218 4426 mini_ranger_kms.cc:227] Using postgres at 127.4.82.148:42071
2026-08-09 01:02:06,255 [I] DB FLAVOR :POSTGRES
2026-08-09 01:02:06,255 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 01:02:06,255 [I] Checking connection
2026-08-09 01:02:06,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 01:02:06,516 [I] connection success
2026-08-09 01:02:06,516 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 01:02:06,516 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 01:02:06,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 01:02:06,746 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 01:02:06,746 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 01:02:06,747 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 01:02:06,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.82.148:42071/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 01:02:07,000 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 01:02:07.449932 4426 mini_ranger_kms.cc:310] Created kms service
I20260809 01:02:07.649609 4426 mini_ranger_kms.cc:326] Created kudu user
I20260809 01:02:07.694321 4426 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 01:02:07 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237325, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:02:08.091946 4426 mini_ranger_kms.cc:384] Added ranger policy
I20260809 01:02:08.092315 4426 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-kms:/tmp/dist-test-taskua6Q0e/build/asan/bin/postgresql.jar:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskua6Q0e/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 01:02:08.092392 4426 mini_ranger_kms.cc:240] Using host: 127.4.82.148
I20260809 01:02:08.112696 4426 mini_ranger_kms.cc:276] Ranger KMS PID: 17535
I20260809 01:02:08.112836 4426 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.82.148:45975
Aug 09, 2026 1:02:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 1:02:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 1:02:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 1:02:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 1:02:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 1:02:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 01:02:10.493705 4426 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.386s user 0.081s sys 0.296s
I20260809 01:02:10.493970 4426 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 01:02:10.494154 4426 mini_ranger_kms.cc:401] 127.4.82.148:45975/kms/v1/keys
Aug 09 01:02:10 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786237330, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.82.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 01:02:11 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237330, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.82.148@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
Aug 09 01:02:11 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237330, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.82.148@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
Aug 09 01:02:13 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:02:13.820878 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41565
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:41565
--ranger_config_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.82.148:45975/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:14.166952 17589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:14.167249 17589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:14.167465 17589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:14.174863 17589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:02:14.174990 17589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:14.175078 17589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:14.175146 17589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:02:14.175215 17589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:02:14.185637 17589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.82.148:45975/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:41565
--ranger_config_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:14.186972 17589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:14.188362 17589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:14.200320 17594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:14.200392 17595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:14.200438 17597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:14.201382 17589 server_base.cc:1034] running on GCE node
I20260809 01:02:14.202147 17589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:14.203609 17589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:14.204826 17589 hybrid_clock.cc:648] HybridClock initialized: now 1786237334204800 us; error 40 us; skew 500 ppm
I20260809 01:02:14.205168 17589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:14 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237334, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:14.208284 17589 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:02:14.209760 17589 webserver.cc:460] Webserver started at http://127.4.82.190:37299/ using document root <none> and password file <none>
I20260809 01:02:14.210366 17589 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:14.210466 17589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:14.210714 17589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 01:02:14 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237334, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:02:14.262823 17589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "e50eb36efa5c4affab88e67397ab8428"
format_stamp: "Formatted at 2026-08-09 01:02:14 on dist-test-slave-07c2"
server_key: "e4b472f409b5d07a8e07302439b4423f"
server_key_iv: "a18217497d9561182511cece89e86472"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:14.263808 17589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "e50eb36efa5c4affab88e67397ab8428"
format_stamp: "Formatted at 2026-08-09 01:02:14 on dist-test-slave-07c2"
server_key: "e4b472f409b5d07a8e07302439b4423f"
server_key_iv: "a18217497d9561182511cece89e86472"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:14.269843 17589 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 01:02:14.287246 17604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:14.288954 17589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 01:02:14.289160 17589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "e50eb36efa5c4affab88e67397ab8428"
format_stamp: "Formatted at 2026-08-09 01:02:14 on dist-test-slave-07c2"
server_key: "e4b472f409b5d07a8e07302439b4423f"
server_key_iv: "a18217497d9561182511cece89e86472"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:14.289407 17589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:14.303406 17589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:14.307463 17589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:14.307792 17589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:14.328058 17589 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:41565
I20260809 01:02:14.328060 17661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:41565 every 8 connection(s)
I20260809 01:02:14.329582 17589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:02:14.329991 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 17589
I20260809 01:02:14.330231 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:02:14.331318 4426 external_mini_cluster.cc:1428] Setting key ce9e58de239ffa50a42d1a0e139e6815
I20260809 01:02:14.334368 17662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:14.349543 17662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428: Bootstrap starting.
I20260809 01:02:14.353682 17662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:14.354925 17662 log.cc:824] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428: Log is configured to *not* fsync() on all Append() calls
Aug 09 01:02:14 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:14.358446 17662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428: No bootstrap required, opened a new log
I20260809 01:02:14.365289 17665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:14.339773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.4.82.190:41565)
0809 01:02:14.345034 (+ 5261us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:14.345047 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:14.345092 (+ 45us) server_negotiation.cc:407] Connection header received
0809 01:02:14.346907 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:14.346933 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:14.347402 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:14.347800 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:14.349270 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:14.350636 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:14.352090 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:14.352758 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:14.352949 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:14.357136 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:14.357145 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:14.357159 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:14.357193 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:14.360401 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:14.361324 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:14.361333 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:14.361348 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:14.361547 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:14.361942 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:14.361945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:14.361947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:14.362478 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:14.362801 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:14.363908 (+ 1107us) server_negotiation.cc:299] Negotiation successful
0809 01:02:14.364247 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5027,"thread_start_us":4722,"threads_started":1}
I20260809 01:02:14.367743 17662 raft_consensus.cc:348] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50eb36efa5c4affab88e67397ab8428" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41565 } }
I20260809 01:02:14.368125 17662 raft_consensus.cc:374] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:14.368263 17662 raft_consensus.cc:729] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e50eb36efa5c4affab88e67397ab8428, State: Initialized, Role: FOLLOWER
I20260809 01:02:14.368872 17662 consensus_queue.cc:260] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50eb36efa5c4affab88e67397ab8428" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41565 } }
I20260809 01:02:14.369122 17662 raft_consensus.cc:388] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:02:14.369259 17662 raft_consensus.cc:482] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:02:14.369442 17662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:14.371834 17662 raft_consensus.cc:504] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50eb36efa5c4affab88e67397ab8428" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41565 } }
I20260809 01:02:14.372324 17662 leader_election.cc:302] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e50eb36efa5c4affab88e67397ab8428; no voters:
I20260809 01:02:14.372926 17662 leader_election.cc:288] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:02:14.373018 17667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:02:14.373755 17667 raft_consensus.cc:686] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 1 LEADER]: Becoming Leader. State: Replica: e50eb36efa5c4affab88e67397ab8428, State: Running, Role: LEADER
I20260809 01:02:14.374321 17667 consensus_queue.cc:237] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50eb36efa5c4affab88e67397ab8428" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41565 } }
I20260809 01:02:14.375059 17662 sys_catalog.cc:564] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:02:14.377297 17668 sys_catalog.cc:455] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e50eb36efa5c4affab88e67397ab8428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50eb36efa5c4affab88e67397ab8428" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41565 } } }
I20260809 01:02:14.377710 17668 sys_catalog.cc:458] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:14.378463 17669 sys_catalog.cc:455] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e50eb36efa5c4affab88e67397ab8428. Latest consensus state: current_term: 1 leader_uuid: "e50eb36efa5c4affab88e67397ab8428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50eb36efa5c4affab88e67397ab8428" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41565 } } }
I20260809 01:02:14.378839 17669 sys_catalog.cc:458] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:14.380352 17672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:02:14.385915 17672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:02:14.386834 17662 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 01:02:14.406822 17672 catalog_manager.cc:1347] Generated new cluster ID: 0b213c8fa0214b1da4f2e7e70eab02a7
I20260809 01:02:14.406980 17672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:02:14.438314 17672 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:02:14.440394 17672 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:02:14.460142 17672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428: Generated new TSK 0
I20260809 01:02:14.461293 17672 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 01:02:15.166790 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.82.148:45975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41565
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 01:02:15 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786237335, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 01:02:15.578542 17708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:15.578953 17708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:15.579181 17708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:15.588754 17708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:15.588871 17708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:15.589046 17708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:02:15.603746 17708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.82.148:45975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:15.605324 17708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:15.606874 17708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:15.625303 17719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:15.632810 17720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:15.637271 17724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:15.639806 17708 server_base.cc:1034] running on GCE node
I20260809 01:02:15.640712 17708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:15.643606 17708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:15.644958 17708 hybrid_clock.cc:648] HybridClock initialized: now 1786237335644844 us; error 103 us; skew 500 ppm
I20260809 01:02:15.645387 17708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:15 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237335, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:15.649933 17708 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:02:15.652194 17708 webserver.cc:460] Webserver started at http://127.4.82.129:41759/ using document root <none> and password file <none>
I20260809 01:02:15.653192 17708 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:15.653345 17708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:15.653743 17708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 01:02:15 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237335, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:02:15.677063 17708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "c4273d36b60f46c6aa42d06b4a8b2578"
format_stamp: "Formatted at 2026-08-09 01:02:15 on dist-test-slave-07c2"
server_key: "a21e3ecd25908043c1509b1e5644f666"
server_key_iv: "081ca76e57d54f2c2278dcb2b443002c"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:15.678311 17708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "c4273d36b60f46c6aa42d06b4a8b2578"
format_stamp: "Formatted at 2026-08-09 01:02:15 on dist-test-slave-07c2"
server_key: "a21e3ecd25908043c1509b1e5644f666"
server_key_iv: "081ca76e57d54f2c2278dcb2b443002c"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:15.686898 17708 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260809 01:02:15.727594 17731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:15.729470 17708 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.004s sys 0.006s
I20260809 01:02:15.729696 17708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "c4273d36b60f46c6aa42d06b4a8b2578"
format_stamp: "Formatted at 2026-08-09 01:02:15 on dist-test-slave-07c2"
server_key: "a21e3ecd25908043c1509b1e5644f666"
server_key_iv: "081ca76e57d54f2c2278dcb2b443002c"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:15.729948 17708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:15.751240 17708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:15.765754 17708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:15.766214 17708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:15.768144 17708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:15.770108 17708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:15.770211 17708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:15.770403 17708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:15.770476 17708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:15.835412 17708 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:33619
I20260809 01:02:15.835421 17852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:33619 every 8 connection(s)
I20260809 01:02:15.837158 17708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:02:15.843786 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 17708
I20260809 01:02:15.844019 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:02:15.844851 4426 external_mini_cluster.cc:1428] Setting key 883414e70fbaaa69eb7ab1347c6edc4c
Aug 09 01:02:15 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237335, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:15.875840 17856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:15.840714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41565 (local address 127.4.82.129:38685)
0809 01:02:15.852920 (+ 12206us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:15.852981 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:15.855329 (+ 2348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:15.856415 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:15.856428 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:15.857072 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:15.857886 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:15.857911 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:15.859474 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:15.859480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:15.861558 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:15.861573 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:15.862848 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:15.862856 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:15.863155 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:15.864011 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:15.864043 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:15.866780 (+ 2737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:15.871031 (+ 4251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:15.871040 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:15.871059 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:15.871691 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:15.872164 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:15.872169 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:15.872173 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:15.872411 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:15.873092 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:15.873100 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:15.873507 (+ 407us) client_negotiation.cc:769] Sending connection context
0809 01:02:15.873813 (+ 306us) client_negotiation.cc:240] Negotiation successful
0809 01:02:15.874366 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11876,"thread_start_us":11346,"threads_started":1}
I20260809 01:02:15.877877 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:15.840176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:38685 (local address 127.4.82.190:41565)
0809 01:02:15.853392 (+ 13216us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:15.853399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:15.853425 (+ 26us) server_negotiation.cc:407] Connection header received
0809 01:02:15.855767 (+ 2342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:15.855774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:15.855876 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:15.855985 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:15.858101 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:15.859266 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:15.861786 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:15.862416 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:15.862545 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:15.867057 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:15.867065 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:15.867070 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:15.867110 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:15.870866 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:15.871859 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:15.871865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:15.871868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:15.872022 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:15.872563 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:15.872568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:15.872571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:15.872927 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:15.877100 (+ 4173us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:15.877252 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 01:02:15.877486 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13079,"thread_start_us":13239,"threads_started":1}
I20260809 01:02:15.879601 17853 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41565
I20260809 01:02:15.879920 17853 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:15.880831 17853 heartbeater.cc:511] Master 127.4.82.190:41565 requested a full tablet report, sending...
I20260809 01:02:15.883388 17623 ts_manager.cc:194] Registered new tserver with Master: c4273d36b60f46c6aa42d06b4a8b2578 (127.4.82.129:33619)
I20260809 01:02:15.885272 17623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:38685
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 01:02:15.925791 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.82.148:45975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41565
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 01:02:15 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237335, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
Aug 09 01:02:16 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237335, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
W20260809 01:02:16.408735 17859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:16.409042 17859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:16.409248 17859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:16.420734 17859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:16.420953 17859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:16.421193 17859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:02:16.436642 17859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.82.148:45975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:16.438601 17859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:16.440539 17859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:16.480749 17866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:16.512142 17867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:16.551779 17859 server_base.cc:1034] running on GCE node
W20260809 01:02:16.552911 17869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:16.553949 17859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:16.557484 17859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:16.558740 17859 hybrid_clock.cc:648] HybridClock initialized: now 1786237336558638 us; error 99 us; skew 500 ppm
I20260809 01:02:16.559178 17859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:16 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:16.563512 17859 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:02:16.584055 17859 webserver.cc:460] Webserver started at http://127.4.82.130:45219/ using document root <none> and password file <none>
I20260809 01:02:16.585165 17859 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:16.585393 17859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:16.585839 17859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 01:02:16 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:02:16.628211 17859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "df7ecdbd199442d98879b706504c5108"
format_stamp: "Formatted at 2026-08-09 01:02:16 on dist-test-slave-07c2"
server_key: "ec202f554dfc6aea0f1dea9e9370eba3"
server_key_iv: "10c83a49fc2225804590b61c19ad0789"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:16.644282 17859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "df7ecdbd199442d98879b706504c5108"
format_stamp: "Formatted at 2026-08-09 01:02:16 on dist-test-slave-07c2"
server_key: "ec202f554dfc6aea0f1dea9e9370eba3"
server_key_iv: "10c83a49fc2225804590b61c19ad0789"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:16.662051 17859 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.004s sys 0.004s
I20260809 01:02:16.695082 17876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:16.708984 17859 fs_manager.cc:725] Time spent opening block manager: real 0.025s user 0.004s sys 0.001s
I20260809 01:02:16.709362 17859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "df7ecdbd199442d98879b706504c5108"
format_stamp: "Formatted at 2026-08-09 01:02:16 on dist-test-slave-07c2"
server_key: "ec202f554dfc6aea0f1dea9e9370eba3"
server_key_iv: "10c83a49fc2225804590b61c19ad0789"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:16.709756 17859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:16.755772 17859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:16.761288 17859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:16.761657 17859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:16.775665 17859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:16.792330 17859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:16.792517 17859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 01:02:16.792799 17859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:16.792943 17859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:16.863015 17859 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:43645
I20260809 01:02:16.863059 18004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:43645 every 8 connection(s)
I20260809 01:02:16.864835 17859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:02:16.867261 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 17859
I20260809 01:02:16.867465 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:02:16.868088 4426 external_mini_cluster.cc:1428] Setting key c60a057f67d640c02537c0b4b95ac189
I20260809 01:02:16.889036 17853 heartbeater.cc:503] Master 127.4.82.190:41565 was elected leader, sending a full tablet report...
Aug 09 01:02:16 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:16.901782 18008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:16.870857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:49889 (local address 127.4.82.190:41565)
0809 01:02:16.881326 (+ 10469us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:16.881334 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:16.881361 (+ 27us) server_negotiation.cc:407] Connection header received
0809 01:02:16.883412 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:16.883418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:16.883536 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:16.883656 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:16.885137 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:16.886360 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:16.888446 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:16.889242 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:16.891401 (+ 2159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:16.894576 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:16.894583 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:16.894588 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:16.894627 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:16.898020 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:16.899024 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:16.899030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:16.899033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:16.899179 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:16.899687 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:16.899691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:16.899694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:16.900047 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:16.900205 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:16.901192 (+ 987us) server_negotiation.cc:299] Negotiation successful
0809 01:02:16.901426 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10329,"thread_start_us":10211,"threads_started":1}
I20260809 01:02:16.903210 18007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:16.870332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41565 (local address 127.4.82.130:49889)
0809 01:02:16.879942 (+ 9610us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:16.880009 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:16.883096 (+ 3087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:16.883919 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:16.883928 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:16.884380 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:16.884956 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:16.884973 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:16.886513 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:16.886518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:16.888248 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:16.888273 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:16.889493 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:16.889501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:16.890135 (+ 634us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:16.891153 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:16.891188 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:16.894311 (+ 3123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:16.898174 (+ 3863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:16.898186 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:16.898205 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:16.898875 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:16.899351 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:16.899358 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:16.899365 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:16.899569 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:16.900209 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:16.900221 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:16.900713 (+ 492us) client_negotiation.cc:769] Sending connection context
0809 01:02:16.901088 (+ 375us) client_negotiation.cc:240] Negotiation successful
0809 01:02:16.901668 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9245,"thread_start_us":9022,"threads_started":1}
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
I20260809 01:02:16.905385 18005 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41565
I20260809 01:02:16.905730 18005 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
I20260809 01:02:16.906595 18005 heartbeater.cc:511] Master 127.4.82.190:41565 requested a full tablet report, sending...
I20260809 01:02:16.908380 17623 ts_manager.cc:194] Registered new tserver with Master: df7ecdbd199442d98879b706504c5108 (127.4.82.130:43645)
I20260809 01:02:16.909529 17623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:49889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 01:02:16.937858 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.82.148:45975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41565
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:17.280406 18011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:17.280728 18011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:17.280860 18011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:17.288348 18011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:17.288465 18011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:17.288592 18011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:02:17.298607 18011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.82.148:45975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:17.299985 18011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:17.301234 18011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:17.314009 18019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:17.314080 18016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:17.314050 18017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:17.315290 18011 server_base.cc:1034] running on GCE node
I20260809 01:02:17.315943 18011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:17.316926 18011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:17.318151 18011 hybrid_clock.cc:648] HybridClock initialized: now 1786237337318105 us; error 41 us; skew 500 ppm
I20260809 01:02:17.318471 18011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:17 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237337, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:17.321624 18011 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:02:17.322988 18011 webserver.cc:460] Webserver started at http://127.4.82.131:40603/ using document root <none> and password file <none>
I20260809 01:02:17.323686 18011 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:17.323792 18011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:17.324059 18011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 01:02:17 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237337, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:02:17.340096 18011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "d974f187c1f4496591619e818ab897c5"
format_stamp: "Formatted at 2026-08-09 01:02:17 on dist-test-slave-07c2"
server_key: "7a80a47d3c21b4389213cc12b8458ab8"
server_key_iv: "eb6d54e3ee1923ef5e335281b54a50e9"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:17.340974 18011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "d974f187c1f4496591619e818ab897c5"
format_stamp: "Formatted at 2026-08-09 01:02:17 on dist-test-slave-07c2"
server_key: "7a80a47d3c21b4389213cc12b8458ab8"
server_key_iv: "eb6d54e3ee1923ef5e335281b54a50e9"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:17.346276 18011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 01:02:17.362342 18027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:17.363981 18011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 01:02:17.364172 18011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "d974f187c1f4496591619e818ab897c5"
format_stamp: "Formatted at 2026-08-09 01:02:17 on dist-test-slave-07c2"
server_key: "7a80a47d3c21b4389213cc12b8458ab8"
server_key_iv: "eb6d54e3ee1923ef5e335281b54a50e9"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:17.364408 18011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:17.393920 18011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:17.398342 18011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:17.398663 18011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:17.399674 18011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:17.401010 18011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:17.401122 18011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:17.401270 18011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:17.401341 18011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:17.435601 18011 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:36755
I20260809 01:02:17.435639 18147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:36755 every 8 connection(s)
I20260809 01:02:17.436906 18011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:02:17.444289 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 18011
I20260809 01:02:17.444494 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:02:17.444988 4426 external_mini_cluster.cc:1428] Setting key 50aa8e57160b9e12b839e638926fa092
Aug 09 01:02:17 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237337, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:17.456305 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:17.439517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:34949 (local address 127.4.82.190:41565)
0809 01:02:17.440066 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:17.440070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:17.440848 (+ 778us) server_negotiation.cc:407] Connection header received
0809 01:02:17.442652 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:17.442658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:17.442763 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:17.442886 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:17.444429 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:17.445303 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:17.447130 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:17.447662 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:17.447803 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:17.450950 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:17.450964 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:17.450967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:17.450999 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:17.453413 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:17.454104 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:17.454112 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:17.454117 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:17.454250 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:17.454688 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:17.454693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:17.454696 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:17.455040 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:17.455217 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:17.455889 (+ 672us) server_negotiation.cc:299] Negotiation successful
0809 01:02:17.456054 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":290,"threads_started":1}
I20260809 01:02:17.457266 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:17.439941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41565 (local address 127.4.82.131:34949)
0809 01:02:17.440683 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:17.440721 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:17.442379 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:17.443127 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:17.443135 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:17.443662 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:17.444256 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:17.444274 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:17.445465 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:17.445491 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:17.446981 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:17.447001 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:17.447891 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:17.447898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:17.448135 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:17.448769 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:17.448795 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:17.450749 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:17.453553 (+ 2804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:17.453562 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:17.453578 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:17.453986 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:17.454362 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:17.454367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:17.454371 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:17.454572 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:17.455205 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:17.455212 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:17.455582 (+ 370us) client_negotiation.cc:769] Sending connection context
0809 01:02:17.455815 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 01:02:17.456183 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":344,"threads_started":1}
I20260809 01:02:17.458794 18148 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41565
I20260809 01:02:17.459124 18148 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:17.459928 18148 heartbeater.cc:511] Master 127.4.82.190:41565 requested a full tablet report, sending...
I20260809 01:02:17.461357 17623 ts_manager.cc:194] Registered new tserver with Master: d974f187c1f4496591619e818ab897c5 (127.4.82.131:36755)
I20260809 01:02:17.462119 17623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:34949
I20260809 01:02:17.472541 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:02:17.474263 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 17708
Aug 09 01:02:17 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 01:02:17.491456 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:17.475147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57642 (local address 127.4.82.129:33619)
0809 01:02:17.475867 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:17.475877 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:17.475907 (+ 30us) server_negotiation.cc:407] Connection header received
0809 01:02:17.475981 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:17.475987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:17.476180 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:17.476313 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:17.477673 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:17.478700 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:17.480330 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:17.480948 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:17.481073 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:17.485745 (+ 4672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:17.485756 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:17.485770 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:17.485817 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:17.488503 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:17.489251 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:17.489258 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:17.489260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:17.489411 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:17.489807 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:17.489811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:17.489813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:17.490149 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:17.490464 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:17.490788 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 01:02:17.491058 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":387,"threads_started":1}
I20260809 01:02:17.492699 17847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:17.575012 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 17708
I20260809 01:02:17.595144 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 17859
Aug 09 01:02:17 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 01:02:17.611975 18156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:17.596107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33880 (local address 127.4.82.130:43645)
0809 01:02:17.596639 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:17.596646 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:17.596669 (+ 23us) server_negotiation.cc:407] Connection header received
0809 01:02:17.596731 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:17.596737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:17.596946 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:17.597115 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:17.598386 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:17.599438 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:17.600813 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:17.601348 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:17.601486 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:17.606077 (+ 4591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:17.606087 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:17.606101 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:17.606133 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:17.608991 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:17.609732 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:17.609741 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:17.609743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:17.609881 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:17.610279 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:17.610285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:17.610288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:17.610622 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:17.610931 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:17.611327 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 01:02:17.611605 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":281,"threads_started":1}
I20260809 01:02:17.613204 18000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:17.694679 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 17859
I20260809 01:02:17.711603 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 18011
Aug 09 01:02:17 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 01:02:17.727363 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:17.712517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52828 (local address 127.4.82.131:36755)
0809 01:02:17.713166 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:17.713173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:17.713201 (+ 28us) server_negotiation.cc:407] Connection header received
0809 01:02:17.713276 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:17.713282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:17.713485 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:17.713660 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:17.715055 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:17.716131 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:17.717438 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:17.717977 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:17.718115 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:17.722431 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:17.722441 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:17.722455 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:17.722487 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:17.725091 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:17.725672 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:17.725679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:17.725681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:17.725797 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:17.726107 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:17.726110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:17.726112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:17.726418 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:17.726646 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:17.726845 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 01:02:17.727022 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":355,"threads_started":1}
I20260809 01:02:17.728637 18143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:17.811970 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 18011
I20260809 01:02:17.831432 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 17589
I20260809 01:02:17.832818 17657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:17.939836 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 17589
I20260809 01:02:17.965026 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41565
--webserver_interface=127.4.82.190
--webserver_port=37299
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:41565
--ranger_config_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.82.148:45975/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.82.148:45975/kms with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:18.307703 18162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:18.308060 18162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:18.308233 18162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:18.316087 18162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:02:18.316262 18162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:18.316380 18162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:18.316484 18162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:02:18.316565 18162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:02:18.326895 18162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.82.148:45975/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:41565
--ranger_config_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=37299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:18.328361 18162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:18.329720 18162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:18.342312 18167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:18.342347 18168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:18.342403 18170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:18.343477 18162 server_base.cc:1034] running on GCE node
I20260809 01:02:18.344291 18162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:18.345676 18162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:18.346882 18162 hybrid_clock.cc:648] HybridClock initialized: now 1786237338346825 us; error 58 us; skew 500 ppm
I20260809 01:02:18.347252 18162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:18 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237338, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:18.350553 18162 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:02:18.352089 18162 webserver.cc:460] Webserver started at http://127.4.82.190:37299/ using document root <none> and password file <none>
I20260809 01:02:18.352788 18162 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:18.352952 18162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 01:02:18 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237338, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:02:18.434576 18162 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 01:02:18.438567 18178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:18.440377 18162 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 01:02:18.440593 18162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "e50eb36efa5c4affab88e67397ab8428"
format_stamp: "Formatted at 2026-08-09 01:02:14 on dist-test-slave-07c2"
server_key: "e4b472f409b5d07a8e07302439b4423f"
server_key_iv: "a18217497d9561182511cece89e86472"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:18.441973 18162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:18.487807 18162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:18.491735 18162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:18.491986 18162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:18.511322 18162 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:41565
I20260809 01:02:18.511330 18231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:41565 every 8 connection(s)
I20260809 01:02:18.512892 18162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:02:18.519672 18232 sys_catalog.cc:263] Verifying existing consensus state
I20260809 01:02:18.522943 18232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428: Bootstrap starting.
I20260809 01:02:18.522969 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 18162
I20260809 01:02:18.523937 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.82.148:45975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:33619
--local_ip_for_outbound_sockets=127.4.82.129
--tserver_master_addrs=127.4.82.190:41565
--webserver_port=41759
--webserver_interface=127.4.82.129
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.82.148:45975/kms with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:02:18.542752 18232 log.cc:824] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:18.552242 18232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 01:02:18.552834 18232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428: Bootstrap complete.
I20260809 01:02:18.562095 18232 raft_consensus.cc:348] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50eb36efa5c4affab88e67397ab8428" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41565 } }
I20260809 01:02:18.563007 18232 raft_consensus.cc:729] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e50eb36efa5c4affab88e67397ab8428, State: Initialized, Role: FOLLOWER
I20260809 01:02:18.563709 18232 consensus_queue.cc:260] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50eb36efa5c4affab88e67397ab8428" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41565 } }
I20260809 01:02:18.563951 18232 raft_consensus.cc:388] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:02:18.564077 18232 raft_consensus.cc:482] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:02:18.564272 18232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:02:18.568629 18232 raft_consensus.cc:504] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50eb36efa5c4affab88e67397ab8428" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41565 } }
I20260809 01:02:18.569131 18232 leader_election.cc:302] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e50eb36efa5c4affab88e67397ab8428; no voters:
I20260809 01:02:18.569828 18232 leader_election.cc:288] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 01:02:18.569967 18237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 01:02:18.571146 18237 raft_consensus.cc:686] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [term 2 LEADER]: Becoming Leader. State: Replica: e50eb36efa5c4affab88e67397ab8428, State: Running, Role: LEADER
I20260809 01:02:18.571822 18237 consensus_queue.cc:237] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50eb36efa5c4affab88e67397ab8428" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41565 } }
I20260809 01:02:18.572283 18232 sys_catalog.cc:564] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:02:18.574808 18238 sys_catalog.cc:455] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e50eb36efa5c4affab88e67397ab8428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50eb36efa5c4affab88e67397ab8428" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41565 } } }
I20260809 01:02:18.575186 18238 sys_catalog.cc:458] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:18.575631 18239 sys_catalog.cc:455] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e50eb36efa5c4affab88e67397ab8428. Latest consensus state: current_term: 2 leader_uuid: "e50eb36efa5c4affab88e67397ab8428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e50eb36efa5c4affab88e67397ab8428" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41565 } } }
I20260809 01:02:18.575941 18239 sys_catalog.cc:458] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428 [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:18.576701 18244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:02:18.586808 18244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:02:18.588438 18244 catalog_manager.cc:1259] Loaded cluster ID: 0b213c8fa0214b1da4f2e7e70eab02a7
I20260809 01:02:18.588624 18244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:02:18.592518 18244 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:02:18.594101 18244 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428: Loaded TSK: 0
I20260809 01:02:18.595736 18244 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:02:18.605078 18232 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260809 01:02:18.986220 18235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:18.986665 18235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:18.986896 18235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:18.998697 18235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:18.998885 18235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:18.999120 18235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:02:19.014619 18235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.82.148:45975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:33619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=41759
--tserver_master_addrs=127.4.82.190:41565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:19.016531 18235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:19.018353 18235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:19.050238 18271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:19.052080 18270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:19.054603 18275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:19.055194 18235 server_base.cc:1034] running on GCE node
I20260809 01:02:19.056005 18235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:19.057299 18235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:19.058751 18235 hybrid_clock.cc:648] HybridClock initialized: now 1786237339058676 us; error 64 us; skew 500 ppm
I20260809 01:02:19.059196 18235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:19 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:19.063726 18235 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:02:19.067026 18235 webserver.cc:460] Webserver started at http://127.4.82.129:41759/ using document root <none> and password file <none>
I20260809 01:02:19.068032 18235 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:19.068208 18235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 01:02:19 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:02:19.091388 18235 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.001s
I20260809 01:02:19.099913 18282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:19.101743 18235 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.003s sys 0.003s
I20260809 01:02:19.101987 18235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "c4273d36b60f46c6aa42d06b4a8b2578"
format_stamp: "Formatted at 2026-08-09 01:02:15 on dist-test-slave-07c2"
server_key: "a21e3ecd25908043c1509b1e5644f666"
server_key_iv: "081ca76e57d54f2c2278dcb2b443002c"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:19.103245 18235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:19.126330 18235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:19.131398 18235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:19.131697 18235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:19.133904 18235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:19.137320 18235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:19.137429 18235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:19.137526 18235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:19.137614 18235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:19.223685 18235 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:33619
I20260809 01:02:19.223785 18398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:33619 every 8 connection(s)
I20260809 01:02:19.225580 18235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:02:19.227154 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 18235
I20260809 01:02:19.228024 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.82.148:45975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:43645
--local_ip_for_outbound_sockets=127.4.82.130
--tserver_master_addrs=127.4.82.190:41565
--webserver_port=45219
--webserver_interface=127.4.82.130
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.82.148:45975/kms with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 01:02:19 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:19.262598 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:19.229694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41565 (local address 127.4.82.129:59535)
0809 01:02:19.232101 (+ 2407us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:19.232147 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:19.234782 (+ 2635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:19.237388 (+ 2606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:19.237402 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:19.238079 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:19.238758 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:19.238781 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:19.240875 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:19.240883 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:19.243092 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:19.243110 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:19.244915 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:19.244925 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:19.245198 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:19.246188 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:19.246219 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:19.252035 (+ 5816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:19.258096 (+ 6061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:19.258103 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:19.258114 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:19.258530 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:19.259009 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:19.259018 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:19.259022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:19.259267 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:19.260102 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:19.260109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:19.260440 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 01:02:19.260707 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 01:02:19.261257 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2107,"thread_start_us":1925,"threads_started":1}
I20260809 01:02:19.262581 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:19.231036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:59535 (local address 127.4.82.190:41565)
0809 01:02:19.235888 (+ 4852us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:19.235910 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:19.235960 (+ 50us) server_negotiation.cc:407] Connection header received
0809 01:02:19.236407 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:19.236426 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:19.236894 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:19.237251 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:19.238986 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:19.240704 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:19.243268 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:19.244347 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:19.244533 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:19.252345 (+ 7812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:19.252356 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:19.252374 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:19.252430 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:19.256397 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:19.258694 (+ 2297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:19.258703 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:19.258711 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:19.258879 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:19.259444 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:19.259449 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:19.259452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:19.259946 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:19.260158 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:19.260933 (+ 775us) server_negotiation.cc:299] Negotiation successful
0809 01:02:19.261328 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4602,"thread_start_us":4387,"threads_started":1}
I20260809 01:02:19.266152 18399 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41565
I20260809 01:02:19.266469 18399 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:19.267233 18399 heartbeater.cc:511] Master 127.4.82.190:41565 requested a full tablet report, sending...
W20260809 01:02:19.270226 18399 heartbeater.cc:644] Failed to heartbeat to 127.4.82.190:41565 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 01:02:19.270450 18399 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:19.270722 18399 heartbeater.cc:511] Master 127.4.82.190:41565 requested a full tablet report, sending...
I20260809 01:02:19.272001 18399 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:19.272317 18399 heartbeater.cc:511] Master 127.4.82.190:41565 requested a full tablet report, sending...
W20260809 01:02:19.273627 18399 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 01:02:19.650298 18403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:19.650662 18403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:19.650870 18403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:19.660164 18403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:19.660310 18403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:19.660506 18403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:02:19.670811 18403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.82.148:45975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:43645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=45219
--tserver_master_addrs=127.4.82.190:41565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:19.672320 18403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:19.674038 18403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:19.691316 18417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:19.691256 18419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:19.697365 18403 server_base.cc:1034] running on GCE node
W20260809 01:02:19.697817 18421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:19.698547 18403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:19.699676 18403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:19.700947 18403 hybrid_clock.cc:648] HybridClock initialized: now 1786237339700892 us; error 49 us; skew 500 ppm
I20260809 01:02:19.701298 18403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:19 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:19.707113 18403 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:02:19.709015 18403 webserver.cc:460] Webserver started at http://127.4.82.130:45219/ using document root <none> and password file <none>
I20260809 01:02:19.709721 18403 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:19.709880 18403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 01:02:19 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:02:19.747097 18403 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.000s sys 0.012s
I20260809 01:02:19.761516 18430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:19.763759 18403 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.007s sys 0.007s
I20260809 01:02:19.764065 18403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "df7ecdbd199442d98879b706504c5108"
format_stamp: "Formatted at 2026-08-09 01:02:16 on dist-test-slave-07c2"
server_key: "ec202f554dfc6aea0f1dea9e9370eba3"
server_key_iv: "10c83a49fc2225804590b61c19ad0789"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:19.765161 18403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:19.788277 18403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:19.793478 18403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:19.793841 18403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:19.799547 18403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:19.801400 18403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:19.801556 18403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 01:02:19.801704 18403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:19.801820 18403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:19.851914 18550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:43645 every 8 connection(s)
I20260809 01:02:19.853673 18403 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:43645
I20260809 01:02:19.855942 18403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:02:19.857573 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 18403
I20260809 01:02:19.858599 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.82.148:45975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:36755
--local_ip_for_outbound_sockets=127.4.82.131
--tserver_master_addrs=127.4.82.190:41565
--webserver_port=40603
--webserver_interface=127.4.82.131
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.82.148:45975/kms with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 01:02:19 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786237339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 01:02:19 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:19.935101 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:19.870544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:36289 (local address 127.4.82.190:41565)
0809 01:02:19.895426 (+ 24882us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:19.895432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:19.898620 (+ 3188us) server_negotiation.cc:407] Connection header received
0809 01:02:19.900964 (+ 2344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:19.900973 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:19.901081 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:19.901177 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:19.903748 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:19.905098 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:19.908311 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:19.909037 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:19.909141 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:19.922657 (+ 13516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:19.922666 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:19.922672 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:19.922713 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:19.929566 (+ 6853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:19.930450 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:19.930455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:19.930458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:19.930600 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:19.931141 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:19.931146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:19.931149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:19.933459 (+ 2310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:19.933594 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:19.934505 (+ 911us) server_negotiation.cc:299] Negotiation successful
0809 01:02:19.934733 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23860,"thread_start_us":24554,"threads_started":1}
I20260809 01:02:19.937328 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:19.869922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41565 (local address 127.4.82.130:36289)
0809 01:02:19.896090 (+ 26168us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:19.896156 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:19.900479 (+ 4323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:19.901811 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:19.901826 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:19.902542 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:19.903506 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:19.903536 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:19.905352 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:19.905359 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:19.908039 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:19.908082 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:19.909567 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:19.909575 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:19.909905 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:19.919472 (+ 9567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:19.919505 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:19.922379 (+ 2874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:19.929741 (+ 7362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:19.929749 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:19.929764 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:19.930311 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:19.930736 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:19.930740 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:19.930744 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:19.930983 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:19.933696 (+ 2713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:19.933703 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:19.934059 (+ 356us) client_negotiation.cc:769] Sending connection context
0809 01:02:19.935567 (+ 1508us) client_negotiation.cc:240] Negotiation successful
0809 01:02:19.936112 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25780,"thread_start_us":28618,"threads_started":1}
I20260809 01:02:19.938982 18551 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41565
I20260809 01:02:19.939332 18551 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:19.940171 18551 heartbeater.cc:511] Master 127.4.82.190:41565 requested a full tablet report, sending...
I20260809 01:02:19.942358 18196 ts_manager.cc:194] Registered new tserver with Master: df7ecdbd199442d98879b706504c5108 (127.4.82.130:43645)
I20260809 01:02:19.944476 18196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:36289
I20260809 01:02:20.273809 18399 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:20.274256 18399 heartbeater.cc:511] Master 127.4.82.190:41565 requested a full tablet report, sending...
I20260809 01:02:20.275651 18196 ts_manager.cc:194] Registered new tserver with Master: c4273d36b60f46c6aa42d06b4a8b2578 (127.4.82.129:33619)
I20260809 01:02:20.276798 18196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:59535
Aug 09 01:02:20 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
W20260809 01:02:20.341902 18553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:20.342257 18553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:20.342408 18553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:20.353432 18553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:20.353571 18553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:20.353775 18553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:02:20.369045 18553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.82.148:45975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:36755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=40603
--tserver_master_addrs=127.4.82.190:41565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:20.370767 18553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:20.372547 18553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:20.411031 18563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:20.415619 18564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:20.417727 18567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:20.418174 18553 server_base.cc:1034] running on GCE node
I20260809 01:02:20.419092 18553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:20.423775 18553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:20.425029 18553 hybrid_clock.cc:648] HybridClock initialized: now 1786237340424960 us; error 54 us; skew 500 ppm
I20260809 01:02:20.425408 18553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:20 dist-test-slave-07c2 krb5kdc[17116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237340, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:20.429459 18553 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:02:20.449710 18553 webserver.cc:460] Webserver started at http://127.4.82.131:40603/ using document root <none> and password file <none>
I20260809 01:02:20.450673 18553 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:20.450856 18553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 01:02:20 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237340, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:02:20.501327 18553 fs_manager.cc:709] Time spent opening directory manager: real 0.020s user 0.006s sys 0.013s
I20260809 01:02:20.510969 18574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:20.512668 18553 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.005s sys 0.005s
I20260809 01:02:20.512831 18553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "d974f187c1f4496591619e818ab897c5"
format_stamp: "Formatted at 2026-08-09 01:02:17 on dist-test-slave-07c2"
server_key: "7a80a47d3c21b4389213cc12b8458ab8"
server_key_iv: "eb6d54e3ee1923ef5e335281b54a50e9"
server_key_version: "kuduclusterkey@0"
I20260809 01:02:20.514137 18553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:20.563591 18553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:20.568594 18553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:20.568910 18553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:20.581681 18553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:20.586076 18553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:20.586207 18553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:20.586326 18553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:20.586412 18553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 01:02:20 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237339, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for HTTP/127.4.82.148@KRBTEST.COM
I20260809 01:02:20.690481 18553 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:36755
I20260809 01:02:20.692414 18553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:02:20.690526 18691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:36755 every 8 connection(s)
I20260809 01:02:20.696298 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 18553
I20260809 01:02:20.729487 18697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:20.699706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51446 (local address 127.4.82.190:41565)
0809 01:02:20.710868 (+ 11162us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:20.710874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:20.710894 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:02:20.710953 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:20.710957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:20.711040 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:20.711164 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:20.713217 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.714304 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.717086 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.717510 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.717664 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:20.721693 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:20.721698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:20.721701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:20.721732 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:20.725584 (+ 3852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:20.726485 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:20.726490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:20.726493 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:20.726680 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:20.727161 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:20.727166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:20.727169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:20.727567 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:20.727853 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:20.728863 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0809 01:02:20.729099 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11032,"thread_start_us":11626,"threads_started":1}
Aug 09 01:02:20 dist-test-slave-07c2 krb5kdc[17116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237340, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:20.741149 18695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:20.696375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:53377 (local address 127.4.82.190:41565)
0809 01:02:20.699765 (+ 3390us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:20.699772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:20.703017 (+ 3245us) server_negotiation.cc:407] Connection header received
0809 01:02:20.703118 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:20.703123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:20.703202 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:20.703324 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:20.724430 (+ 21106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.725694 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.728192 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.728873 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.728964 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:20.734005 (+ 5041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:20.734014 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:20.734020 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:20.734065 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:20.737462 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:20.738368 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:20.738373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:20.738376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:20.738512 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:20.739065 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:20.739070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:20.739073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:20.739458 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:20.739584 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:20.740526 (+ 942us) server_negotiation.cc:299] Negotiation successful
0809 01:02:20.740756 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3251,"thread_start_us":3138,"threads_started":1}
I20260809 01:02:20.744112 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:20.695847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41565 (local address 127.4.82.131:53377)
0809 01:02:20.700361 (+ 4514us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:20.700417 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:20.702692 (+ 2275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:20.703758 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:20.703769 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:20.704459 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:20.724177 (+ 19718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:20.724214 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.725924 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.725929 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:20.728022 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:20.728046 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.729321 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.729328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:20.729601 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:20.730326 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:20.730355 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:20.733658 (+ 3303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:20.737619 (+ 3961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:20.737628 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:20.737646 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:20.738229 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:20.738659 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:20.738664 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:20.738679 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:20.738932 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:20.739685 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:20.739693 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:20.740105 (+ 412us) client_negotiation.cc:769] Sending connection context
0809 01:02:20.741616 (+ 1511us) client_negotiation.cc:240] Negotiation successful
0809 01:02:20.742165 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4223,"thread_start_us":4009,"threads_started":1}
I20260809 01:02:20.746340 18692 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41565
I20260809 01:02:20.746636 18692 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:20.752272 18692 heartbeater.cc:511] Master 127.4.82.190:41565 requested a full tablet report, sending...
I20260809 01:02:20.754017 18196 ts_manager.cc:194] Registered new tserver with Master: d974f187c1f4496591619e818ab897c5 (127.4.82.131:36755)
I20260809 01:02:20.755131 18196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:53377
I20260809 01:02:20.763042 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:02:20.847564 18695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:20.829536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51450 (local address 127.4.82.190:41565)
0809 01:02:20.829778 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:20.829785 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:20.829814 (+ 29us) server_negotiation.cc:407] Connection header received
0809 01:02:20.830010 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:20.830017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:20.830129 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:20.830208 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:20.832494 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.833607 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.835761 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.836417 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.836535 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:20.840428 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:20.840435 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:20.840439 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:20.840480 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:20.844050 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:20.844894 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:20.844899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:20.844902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:20.845061 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:20.845584 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:20.845590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:20.845593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:20.845989 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:20.846231 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:20.846936 (+ 705us) server_negotiation.cc:299] Negotiation successful
0809 01:02:20.847189 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 01:02:20.854843 18196 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:02:20.865147 18196 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:02:20.928970 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:20.885755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:36755 (local address 127.0.0.1:60084)
0809 01:02:20.918096 (+ 32341us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:20.918153 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:20.918382 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:20.920791 (+ 2409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:20.920798 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:20.920829 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:20.921276 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:20.921289 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.923438 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.923443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:20.925740 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:20.925758 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.927990 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.927999 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:20.928214 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:20.928277 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 01:02:20.928400 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 01:02:20.928610 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30585,"thread_start_us":32884,"threads_started":1}
I20260809 01:02:20.930584 18709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:20.886383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.4.82.131:36755)
0809 01:02:20.920282 (+ 33899us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:20.920288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:20.920315 (+ 27us) server_negotiation.cc:407] Connection header received
0809 01:02:20.920393 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:20.920398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:20.920575 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:20.920787 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:20.921518 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.923255 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.925965 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.927483 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.927611 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:20.927830 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:20.930062 (+ 2232us) server_negotiation.cc:299] Negotiation successful
0809 01:02:20.930237 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33705,"thread_start_us":33601,"threads_started":1}
I20260809 01:02:20.930850 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:20.885768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:33619 (local address 127.0.0.1:47802)
0809 01:02:20.921286 (+ 35518us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:20.921305 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:20.921519 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:20.922441 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:20.922445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:20.922470 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:20.922839 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:20.922851 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.924841 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.924846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:20.927914 (+ 3068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:20.927931 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.930064 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.930071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:20.930233 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:20.930264 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 01:02:20.930353 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 01:02:20.930476 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35379,"mutex_wait_us":3,"thread_start_us":33931,"threads_started":1}
I20260809 01:02:20.931614 18710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:20.886740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47802 (local address 127.4.82.129:33619)
0809 01:02:20.915738 (+ 28998us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:20.915752 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:20.921410 (+ 5658us) server_negotiation.cc:407] Connection header received
0809 01:02:20.921738 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:20.921747 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:20.921978 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:20.922152 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:20.923023 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.924657 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.928097 (+ 3440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.929525 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.929664 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:20.929879 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:20.931128 (+ 1249us) server_negotiation.cc:299] Negotiation successful
0809 01:02:20.931282 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28725,"thread_start_us":29014,"threads_started":1}
I20260809 01:02:20.934890 18624 tablet_service.cc:1467] Processing CreateTablet for tablet 07eef0b8441346dfa697663b2678a23c (DEFAULT_TABLE table=test-table [id=56b459e92901434d97eca6b461cfc53b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:02:20.935453 18331 tablet_service.cc:1467] Processing CreateTablet for tablet 07eef0b8441346dfa697663b2678a23c (DEFAULT_TABLE table=test-table [id=56b459e92901434d97eca6b461cfc53b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:02:20.936656 18624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07eef0b8441346dfa697663b2678a23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:20.936895 18331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07eef0b8441346dfa697663b2678a23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:20.939796 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:20.887985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:43645 (local address 127.0.0.1:34256)
0809 01:02:20.929779 (+ 41794us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:20.929797 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:20.929893 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:20.932985 (+ 3092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:20.932990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:20.933018 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:20.933180 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:20.933193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.935228 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.935234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:20.937005 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:20.937023 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.939024 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.939032 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:20.939241 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:20.939274 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 01:02:20.939365 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 01:02:20.939504 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41706,"thread_start_us":50762,"threads_started":1}
I20260809 01:02:20.940912 18712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:20.888476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34256 (local address 127.4.82.130:43645)
0809 01:02:20.931862 (+ 43386us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:20.931871 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:20.931902 (+ 31us) server_negotiation.cc:407] Connection header received
0809 01:02:20.931999 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:20.932008 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:20.932222 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:20.933356 (+ 1134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:20.933579 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.935062 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.937220 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:20.938632 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:20.940149 (+ 1517us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:20.940367 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:20.940434 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 01:02:20.940584 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43200,"thread_start_us":43921,"threads_started":1}
I20260809 01:02:20.944336 18483 tablet_service.cc:1467] Processing CreateTablet for tablet 07eef0b8441346dfa697663b2678a23c (DEFAULT_TABLE table=test-table [id=56b459e92901434d97eca6b461cfc53b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:02:20.945727 18483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07eef0b8441346dfa697663b2678a23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:20.949106 18551 heartbeater.cc:503] Master 127.4.82.190:41565 was elected leader, sending a full tablet report...
I20260809 01:02:20.963665 18713 tablet_bootstrap.cc:492] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578: Bootstrap starting.
I20260809 01:02:20.966931 18713 tablet_bootstrap.cc:654] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:20.967978 18714 tablet_bootstrap.cc:492] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108: Bootstrap starting.
I20260809 01:02:20.968369 18713 log.cc:824] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:20.975612 18714 tablet_bootstrap.cc:654] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:20.977171 18714 log.cc:824] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:20.980279 18716 tablet_bootstrap.cc:492] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5: Bootstrap starting.
I20260809 01:02:20.984807 18716 tablet_bootstrap.cc:654] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:20.988597 18713 tablet_bootstrap.cc:492] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578: No bootstrap required, opened a new log
I20260809 01:02:20.988639 18716 log.cc:824] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:20.989081 18713 ts_tablet_manager.cc:1397] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.017s
I20260809 01:02:20.993170 18714 tablet_bootstrap.cc:492] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108: No bootstrap required, opened a new log
I20260809 01:02:20.993484 18714 ts_tablet_manager.cc:1397] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.010s
I20260809 01:02:20.999373 18716 tablet_bootstrap.cc:492] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5: No bootstrap required, opened a new log
I20260809 01:02:20.999679 18716 ts_tablet_manager.cc:1397] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.000s
I20260809 01:02:21.003643 18713 raft_consensus.cc:348] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d974f187c1f4496591619e818ab897c5" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 36755 } } peers { permanent_uuid: "c4273d36b60f46c6aa42d06b4a8b2578" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 33619 } } peers { permanent_uuid: "df7ecdbd199442d98879b706504c5108" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43645 } }
I20260809 01:02:21.004256 18713 raft_consensus.cc:374] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:21.004411 18713 raft_consensus.cc:729] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4273d36b60f46c6aa42d06b4a8b2578, State: Initialized, Role: FOLLOWER
I20260809 01:02:21.003948 18714 raft_consensus.cc:348] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d974f187c1f4496591619e818ab897c5" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 36755 } } peers { permanent_uuid: "c4273d36b60f46c6aa42d06b4a8b2578" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 33619 } } peers { permanent_uuid: "df7ecdbd199442d98879b706504c5108" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43645 } }
I20260809 01:02:21.004647 18714 raft_consensus.cc:374] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:21.004773 18714 raft_consensus.cc:729] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df7ecdbd199442d98879b706504c5108, State: Initialized, Role: FOLLOWER
I20260809 01:02:21.005295 18713 consensus_queue.cc:260] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d974f187c1f4496591619e818ab897c5" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 36755 } } peers { permanent_uuid: "c4273d36b60f46c6aa42d06b4a8b2578" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 33619 } } peers { permanent_uuid: "df7ecdbd199442d98879b706504c5108" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43645 } }
I20260809 01:02:21.005375 18714 consensus_queue.cc:260] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d974f187c1f4496591619e818ab897c5" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 36755 } } peers { permanent_uuid: "c4273d36b60f46c6aa42d06b4a8b2578" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 33619 } } peers { permanent_uuid: "df7ecdbd199442d98879b706504c5108" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43645 } }
I20260809 01:02:21.014032 18713 ts_tablet_manager.cc:1428] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578: Time spent starting tablet: real 0.025s user 0.015s sys 0.004s
I20260809 01:02:21.016877 18714 ts_tablet_manager.cc:1428] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108: Time spent starting tablet: real 0.023s user 0.020s sys 0.000s
I20260809 01:02:21.017856 18716 raft_consensus.cc:348] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d974f187c1f4496591619e818ab897c5" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 36755 } } peers { permanent_uuid: "c4273d36b60f46c6aa42d06b4a8b2578" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 33619 } } peers { permanent_uuid: "df7ecdbd199442d98879b706504c5108" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43645 } }
I20260809 01:02:21.018325 18716 raft_consensus.cc:374] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:21.018432 18716 raft_consensus.cc:729] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d974f187c1f4496591619e818ab897c5, State: Initialized, Role: FOLLOWER
I20260809 01:02:21.018620 18399 heartbeater.cc:503] Master 127.4.82.190:41565 was elected leader, sending a full tablet report...
I20260809 01:02:21.023840 18716 consensus_queue.cc:260] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d974f187c1f4496591619e818ab897c5" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 36755 } } peers { permanent_uuid: "c4273d36b60f46c6aa42d06b4a8b2578" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 33619 } } peers { permanent_uuid: "df7ecdbd199442d98879b706504c5108" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43645 } }
I20260809 01:02:21.032577 18692 heartbeater.cc:503] Master 127.4.82.190:41565 was elected leader, sending a full tablet report...
I20260809 01:02:21.033411 18716 ts_tablet_manager.cc:1428] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5: Time spent starting tablet: real 0.034s user 0.012s sys 0.008s
W20260809 01:02:21.115173 18552 tablet.cc:2369] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 01:02:21.197829 18693 tablet.cc:2369] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:02:21.226447 18722 raft_consensus.cc:482] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:02:21.226761 18722 raft_consensus.cc:504] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d974f187c1f4496591619e818ab897c5" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 36755 } } peers { permanent_uuid: "c4273d36b60f46c6aa42d06b4a8b2578" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 33619 } } peers { permanent_uuid: "df7ecdbd199442d98879b706504c5108" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43645 } }
I20260809 01:02:21.228909 18722 leader_election.cc:288] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4273d36b60f46c6aa42d06b4a8b2578 (127.4.82.129:33619), df7ecdbd199442d98879b706504c5108 (127.4.82.130:43645)
W20260809 01:02:21.237895 18400 tablet.cc:2369] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:02:21.238299 18694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:21.229155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:33619 (local address 127.4.82.131:43611)
0809 01:02:21.229443 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:21.229467 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:21.229596 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:21.231859 (+ 2263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:21.231864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:21.231896 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:21.232160 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:21.232171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.233543 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.233548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:21.235988 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:21.236000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.237176 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.237182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:21.237361 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:21.237385 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 01:02:21.237495 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 01:02:21.237998 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":203}
I20260809 01:02:21.238296 18712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:21.230034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:60895 (local address 127.4.82.130:43645)
0809 01:02:21.230286 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:21.230294 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:21.230918 (+ 624us) server_negotiation.cc:407] Connection header received
0809 01:02:21.231182 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:21.231188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:21.231278 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:21.231368 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:21.232172 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.233325 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.236167 (+ 2842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.237341 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.237506 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:21.237574 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:21.237949 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 01:02:21.238070 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 01:02:21.239001 18731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:21.229238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:43645 (local address 127.4.82.131:60895)
0809 01:02:21.230819 (+ 1581us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:21.230836 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:21.231022 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:21.231564 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:21.231569 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:21.231592 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:21.231978 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:21.231989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.233544 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.233548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:21.235986 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:21.236000 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.237567 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.237573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:21.237759 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:21.237794 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 01:02:21.238517 (+ 723us) client_negotiation.cc:240] Negotiation successful
0809 01:02:21.238637 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1465,"mutex_wait_us":116,"thread_start_us":1258,"threads_started":1}
I20260809 01:02:21.239228 18710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:21.231237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:43611 (local address 127.4.82.129:33619)
0809 01:02:21.231457 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:21.231462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:21.231481 (+ 19us) server_negotiation.cc:407] Connection header received
0809 01:02:21.231547 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:21.231551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:21.231622 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:21.231712 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:21.232326 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.233376 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.236195 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.237014 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.237178 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:21.237251 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:21.238835 (+ 1584us) server_negotiation.cc:299] Negotiation successful
0809 01:02:21.238969 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 01:02:21.240077 18506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07eef0b8441346dfa697663b2678a23c" candidate_uuid: "d974f187c1f4496591619e818ab897c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df7ecdbd199442d98879b706504c5108" is_pre_election: true
I20260809 01:02:21.240132 18353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07eef0b8441346dfa697663b2678a23c" candidate_uuid: "d974f187c1f4496591619e818ab897c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4273d36b60f46c6aa42d06b4a8b2578" is_pre_election: true
I20260809 01:02:21.240597 18506 raft_consensus.cc:2454] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d974f187c1f4496591619e818ab897c5 in term 0.
I20260809 01:02:21.240628 18353 raft_consensus.cc:2454] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d974f187c1f4496591619e818ab897c5 in term 0.
I20260809 01:02:21.241240 18576 leader_election.cc:302] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d974f187c1f4496591619e818ab897c5, df7ecdbd199442d98879b706504c5108; no voters:
I20260809 01:02:21.241634 18722 raft_consensus.cc:2781] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:02:21.241791 18722 raft_consensus.cc:482] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:02:21.241896 18722 raft_consensus.cc:3037] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:21.244135 18722 raft_consensus.cc:504] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d974f187c1f4496591619e818ab897c5" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 36755 } } peers { permanent_uuid: "c4273d36b60f46c6aa42d06b4a8b2578" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 33619 } } peers { permanent_uuid: "df7ecdbd199442d98879b706504c5108" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43645 } }
I20260809 01:02:21.244726 18722 leader_election.cc:288] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [CANDIDATE]: Term 1 election: Requested vote from peers c4273d36b60f46c6aa42d06b4a8b2578 (127.4.82.129:33619), df7ecdbd199442d98879b706504c5108 (127.4.82.130:43645)
I20260809 01:02:21.245177 18353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07eef0b8441346dfa697663b2678a23c" candidate_uuid: "d974f187c1f4496591619e818ab897c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4273d36b60f46c6aa42d06b4a8b2578"
I20260809 01:02:21.245275 18506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07eef0b8441346dfa697663b2678a23c" candidate_uuid: "d974f187c1f4496591619e818ab897c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df7ecdbd199442d98879b706504c5108"
I20260809 01:02:21.245374 18353 raft_consensus.cc:3037] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:21.245479 18506 raft_consensus.cc:3037] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:21.247974 18353 raft_consensus.cc:2454] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d974f187c1f4496591619e818ab897c5 in term 1.
I20260809 01:02:21.248409 18578 leader_election.cc:302] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4273d36b60f46c6aa42d06b4a8b2578, d974f187c1f4496591619e818ab897c5; no voters:
I20260809 01:02:21.248517 18506 raft_consensus.cc:2454] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d974f187c1f4496591619e818ab897c5 in term 1.
I20260809 01:02:21.248710 18722 raft_consensus.cc:2781] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:02:21.249266 18722 raft_consensus.cc:686] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [term 1 LEADER]: Becoming Leader. State: Replica: d974f187c1f4496591619e818ab897c5, State: Running, Role: LEADER
I20260809 01:02:21.249786 18722 consensus_queue.cc:237] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d974f187c1f4496591619e818ab897c5" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 36755 } } peers { permanent_uuid: "c4273d36b60f46c6aa42d06b4a8b2578" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 33619 } } peers { permanent_uuid: "df7ecdbd199442d98879b706504c5108" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43645 } }
I20260809 01:02:21.254142 18194 catalog_manager.cc:5495] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to d974f187c1f4496591619e818ab897c5 (127.4.82.131). New cstate: current_term: 1 leader_uuid: "d974f187c1f4496591619e818ab897c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d974f187c1f4496591619e818ab897c5" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 36755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4273d36b60f46c6aa42d06b4a8b2578" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 33619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df7ecdbd199442d98879b706504c5108" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43645 } health_report { overall_health: UNKNOWN } } }
I20260809 01:02:21.356961 18709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:21.351806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.4.82.131:36755)
0809 01:02:21.351992 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:21.351996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:21.352301 (+ 305us) server_negotiation.cc:407] Connection header received
0809 01:02:21.352472 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:21.352475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:21.352555 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:21.352689 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:02:21.353100 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.353810 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.355129 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.355522 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.355611 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:21.356029 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:02:21.356490 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:02:21.356603 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:21.356710 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 01:02:21.356772 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 01:02:21.365090 18506 raft_consensus.cc:1261] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108 [term 1 FOLLOWER]: Refusing update from remote peer d974f187c1f4496591619e818ab897c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:02:21.365090 18353 raft_consensus.cc:1261] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578 [term 1 FOLLOWER]: Refusing update from remote peer d974f187c1f4496591619e818ab897c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:02:21.366098 18732 consensus_queue.cc:1035] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4273d36b60f46c6aa42d06b4a8b2578" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 33619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:02:21.366500 18722 consensus_queue.cc:1035] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df7ecdbd199442d98879b706504c5108" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:02:21.397696 18196 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:02:21.398007 18196 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:02:21.401356 18196 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e50eb36efa5c4affab88e67397ab8428: Sending DeleteTablet for 3 replicas of tablet 07eef0b8441346dfa697663b2678a23c
I20260809 01:02:21.402506 18331 tablet_service.cc:1514] Processing DeleteTablet for tablet 07eef0b8441346dfa697663b2678a23c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:02:21 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:47802
I20260809 01:02:21.402509 18624 tablet_service.cc:1514] Processing DeleteTablet for tablet 07eef0b8441346dfa697663b2678a23c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:02:21 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:60084
I20260809 01:02:21.402796 18483 tablet_service.cc:1514] Processing DeleteTablet for tablet 07eef0b8441346dfa697663b2678a23c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:02:21 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:34256
I20260809 01:02:21.406399 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 18235
I20260809 01:02:21.415511 18749 tablet_replica.cc:330] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578: stopping tablet replica
I20260809 01:02:21.416020 18752 tablet_replica.cc:330] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108: stopping tablet replica
I20260809 01:02:21.416213 18749 raft_consensus.cc:2229] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:02:21.416431 18752 raft_consensus.cc:2229] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:02:21.416836 18749 raft_consensus.cc:2258] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:02:21.416855 18752 raft_consensus.cc:2258] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:02:21.419224 18749 ts_tablet_manager.cc:1905] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:02:21.419374 18751 tablet_replica.cc:330] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5: stopping tablet replica
I20260809 01:02:21.419823 18751 raft_consensus.cc:2229] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:02:21.420572 18751 raft_consensus.cc:2258] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:02:21.422132 18752 ts_tablet_manager.cc:1905] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:02:21.424060 18751 ts_tablet_manager.cc:1905] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:02:21.426481 18749 ts_tablet_manager.cc:1918] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:02:21.426679 18749 log.cc:1196] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/07eef0b8441346dfa697663b2678a23c
I20260809 01:02:21.427238 18749 ts_tablet_manager.cc:1939] T 07eef0b8441346dfa697663b2678a23c P c4273d36b60f46c6aa42d06b4a8b2578: Deleting consensus metadata
I20260809 01:02:21.430181 18752 ts_tablet_manager.cc:1918] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:02:21.430394 18752 log.cc:1196] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/07eef0b8441346dfa697663b2678a23c
I20260809 01:02:21.430994 18752 ts_tablet_manager.cc:1939] T 07eef0b8441346dfa697663b2678a23c P df7ecdbd199442d98879b706504c5108: Deleting consensus metadata
I20260809 01:02:21.431115 18182 catalog_manager.cc:4841] TS c4273d36b60f46c6aa42d06b4a8b2578 (127.4.82.129:33619): tablet 07eef0b8441346dfa697663b2678a23c (table test-table [id=56b459e92901434d97eca6b461cfc53b]) successfully deleted
I20260809 01:02:21.432693 18180 catalog_manager.cc:4841] TS df7ecdbd199442d98879b706504c5108 (127.4.82.130:43645): tablet 07eef0b8441346dfa697663b2678a23c (table test-table [id=56b459e92901434d97eca6b461cfc53b]) successfully deleted
I20260809 01:02:21.432708 18751 ts_tablet_manager.cc:1918] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:02:21.432894 18751 log.cc:1196] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/07eef0b8441346dfa697663b2678a23c
I20260809 01:02:21.433478 18751 ts_tablet_manager.cc:1939] T 07eef0b8441346dfa697663b2678a23c P d974f187c1f4496591619e818ab897c5: Deleting consensus metadata
I20260809 01:02:21.435132 18179 catalog_manager.cc:4841] TS d974f187c1f4496591619e818ab897c5 (127.4.82.131:36755): tablet 07eef0b8441346dfa697663b2678a23c (table test-table [id=56b459e92901434d97eca6b461cfc53b]) successfully deleted
I20260809 01:02:21.444113 18710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:21.408641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47804 (local address 127.4.82.129:33619)
0809 01:02:21.408869 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:21.408878 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:21.415406 (+ 6528us) server_negotiation.cc:407] Connection header received
0809 01:02:21.417469 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:21.417476 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:21.417556 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:21.417636 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:21.421756 (+ 4120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.422872 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.433505 (+ 10633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.433992 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.434135 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:21.436942 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:21.436957 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:21.436976 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:21.437023 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:21.441067 (+ 4044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:21.441784 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:21.441795 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:21.441799 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:21.441963 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:21.442343 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:21.442348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:21.442351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:21.442833 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:21.443159 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:21.443440 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 01:02:21.443679 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 01:02:21.444806 18394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:21.580838 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 18235
I20260809 01:02:21.600633 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 18403
I20260809 01:02:21.616673 18712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:21.601455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.4.82.130:43645)
0809 01:02:21.601635 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:21.601639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:21.601704 (+ 65us) server_negotiation.cc:407] Connection header received
0809 01:02:21.601898 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:21.601902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:21.601985 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:21.602081 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:21.603935 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.604618 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.606535 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.606940 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.607037 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:21.610571 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:21.610590 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:21.610608 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:21.610640 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:21.613539 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:21.614334 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:21.614343 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:21.614345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:21.614486 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:21.614897 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:21.614901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:21.614903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:21.615236 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:21.615733 (+ 497us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:21.616104 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 01:02:21.616267 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 01:02:21.617280 18546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:21.705781 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 18403
I20260809 01:02:21.726217 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 18553
I20260809 01:02:21.742908 18709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:21.726972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.4.82.131:36755)
0809 01:02:21.727173 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:21.727177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:21.727259 (+ 82us) server_negotiation.cc:407] Connection header received
0809 01:02:21.727470 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:21.727473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:21.727552 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:21.727640 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:21.729637 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.730446 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.732431 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:21.732909 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:21.733001 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:21.736560 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:21.736571 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:21.736589 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:21.736624 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:21.740036 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:21.740875 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:21.740882 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:21.740884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:21.741018 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:21.741417 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:21.741420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:21.741422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:21.741770 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:21.741979 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:21.742380 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 01:02:21.742535 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 01:02:21.743752 18687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:21.865101 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 18553
I20260809 01:02:21.886266 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 18162
I20260809 01:02:21.887545 18227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:22.002818 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 18162
I20260809 01:02:22.028710 4426 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 01:02:22.363812 4426 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 01:02:22.364007 4426 mini_ranger.cc:66] Stopping Ranger...
I20260809 01:02:22.709450 4426 mini_ranger.cc:68] Stopped Ranger
2026-08-09 01:02:22.709 UTC [17148] LOG: received smart shutdown request
2026-08-09 01:02:22.713 UTC [17148] LOG: background worker "logical replication launcher" (PID 17158) exited with exit code 1
2026-08-09 01:02:22.713 UTC [17152] LOG: shutting down
2026-08-09 01:02:22.857 UTC [17148] LOG: database system is shut down
2026-08-09T01:02:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52330 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:02:23 dist-test-slave-07c2 krb5kdc[18766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:02:23 dist-test-slave-07c2 krb5kdc[18766](info): set up 2 sockets
Aug 09 01:02:23 dist-test-slave-07c2 krb5kdc[18766](info): commencing operation
krb5kdc: starting...
W20260809 01:02:24.139981 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:02:24 dist-test-slave-07c2 krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:02:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:02:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:02:24.400887 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:33771
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:44403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:33771,127.4.82.189:45051,127.4.82.188:32775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:24.729724 18780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:24.730026 18780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:24.730111 18780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:24.736955 18780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:02:24.737046 18780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:24.737090 18780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:24.737145 18780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:02:24.737197 18780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:02:24.749115 18780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:44403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:33771,127.4.82.189:45051,127.4.82.188:32775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:33771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:24.750702 18780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:24.752174 18780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:24.763800 18785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:24.763804 18789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:24.763772 18787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:24.765457 18780 server_base.cc:1034] running on GCE node
I20260809 01:02:24.766219 18780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:24.767498 18780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:24.768704 18780 hybrid_clock.cc:648] HybridClock initialized: now 1786237344768652 us; error 43 us; skew 500 ppm
I20260809 01:02:24.769069 18780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:24 dist-test-slave-07c2 krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237344, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:24.772176 18780 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:02:24.773692 18780 webserver.cc:460] Webserver started at http://127.4.82.190:39811/ using document root <none> and password file <none>
I20260809 01:02:24.774380 18780 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:24.774525 18780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:24.774832 18780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:02:24.777787 18780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "8fceeb58d148444bb0218eea82f0451b"
format_stamp: "Formatted at 2026-08-09 01:02:24 on dist-test-slave-07c2"
server_key: "d7fc01b8b5fd8f22b324afaa9e48edd0"
server_key_iv: "64bdcf8888e20ad4dcd2ddb5c87fedc9"
server_key_version: "encryptionkey@0"
I20260809 01:02:24.778625 18780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "8fceeb58d148444bb0218eea82f0451b"
format_stamp: "Formatted at 2026-08-09 01:02:24 on dist-test-slave-07c2"
server_key: "d7fc01b8b5fd8f22b324afaa9e48edd0"
server_key_iv: "64bdcf8888e20ad4dcd2ddb5c87fedc9"
server_key_version: "encryptionkey@0"
I20260809 01:02:24.783660 18780 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 01:02:24.786952 18796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:24.788295 18780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:02:24.788470 18780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "8fceeb58d148444bb0218eea82f0451b"
format_stamp: "Formatted at 2026-08-09 01:02:24 on dist-test-slave-07c2"
server_key: "d7fc01b8b5fd8f22b324afaa9e48edd0"
server_key_iv: "64bdcf8888e20ad4dcd2ddb5c87fedc9"
server_key_version: "encryptionkey@0"
I20260809 01:02:24.788687 18780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:24.805629 18780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:24.809321 18780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:24.809598 18780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:24.827710 18780 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:33771
I20260809 01:02:24.827782 18851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:33771 every 8 connection(s)
I20260809 01:02:24.829056 18780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:02:24.832772 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 18780
I20260809 01:02:24.832914 18852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:24.833016 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:02:24.833889 4426 external_mini_cluster.cc:1428] Setting key fdd62b929fd7a508990e8580b462c7fa
I20260809 01:02:24.843537 18852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:02:24 dist-test-slave-07c2 krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237344, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.82.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.189@KRBTEST.COM" created.
I20260809 01:02:24.868734 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:24.847398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33771 (local address 127.0.0.1:48504)
0809 01:02:24.848328 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:24.848385 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:24.850386 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:24.851178 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:24.851184 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:24.851931 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:24.852258 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:24.852275 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:24.854084 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:24.854092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:24.856189 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:24.856214 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:24.857289 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:24.857298 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:24.857533 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:24.858294 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:24.858338 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:24.861043 (+ 2705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:24.864596 (+ 3553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:24.864609 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:24.864620 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:24.864992 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:24.865387 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:24.865394 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:24.865397 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:24.865546 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:24.866268 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:24.866280 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:24.866484 (+ 204us) client_negotiation.cc:769] Sending connection context
0809 01:02:24.866717 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 01:02:24.867383 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":171,"thread_start_us":440,"threads_started":1}
I20260809 01:02:24.868731 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:24.847399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.4.82.190:33771)
0809 01:02:24.849172 (+ 1773us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:24.849180 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:24.849229 (+ 49us) server_negotiation.cc:407] Connection header received
0809 01:02:24.850753 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:24.850758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:24.850958 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:24.851172 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:24.852518 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:24.853909 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:24.856355 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:24.857026 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:24.857161 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:24.861343 (+ 4182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:24.861357 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:24.861373 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:24.861425 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:24.864465 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:24.865132 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:24.865137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:24.865140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:24.865280 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:24.865659 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:24.865664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:24.865669 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:24.866128 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:24.866344 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:24.866860 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 01:02:24.867250 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":1543,"spinlock_wait_cycles":30976,"thread_start_us":1331,"threads_started":1}
I20260809 01:02:24.871842 18852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:02:24.872905 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:24.872531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:45051 (local address 127.0.0.1:46936)
0809 01:02:24.872699 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:24.872801 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 01:02:24.873138 18800 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
I20260809 01:02:24.874675 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:24.874336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:45051 (local address 127.0.0.1:46938)
0809 01:02:24.874481 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:24.874548 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 01:02:24.874866 18852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:45051: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.189.keytab.
Entry for principal kudu/127.4.82.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.189.keytab.
I20260809 01:02:24.892571 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.189:45051
--webserver_interface=127.4.82.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.189.keytab
--principal=kudu/127.4.82.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:44403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:33771,127.4.82.189:45051,127.4.82.188:32775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:02:24.924165 18852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } attempt: 1
I20260809 01:02:24.925262 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:24.924827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:45051 (local address 127.0.0.1:46944)
0809 01:02:24.925025 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:24.925128 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 01:02:24.926235 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:24.925872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:45051 (local address 127.0.0.1:46946)
0809 01:02:24.926015 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:24.926085 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 01:02:24.926427 18852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:45051: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
I20260809 01:02:24.994653 18852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } attempt: 2
I20260809 01:02:24.995795 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:24.995363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:45051 (local address 127.0.0.1:46950)
0809 01:02:24.995552 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:24.995650 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 01:02:24.996860 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:24.996437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:45051 (local address 127.0.0.1:46954)
0809 01:02:24.996581 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:24.996663 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 01:02:24.997057 18852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:45051: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
I20260809 01:02:25.088304 18852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } attempt: 3
I20260809 01:02:25.089419 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.088967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:45051 (local address 127.0.0.1:46962)
0809 01:02:25.089174 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.089288 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 01:02:25.090550 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.090124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:45051 (local address 127.0.0.1:46964)
0809 01:02:25.090271 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.090375 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 01:02:25.090731 18852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:45051: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
W20260809 01:02:25.215919 18860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:25.216257 18860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:25.216427 18860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:25.223589 18860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:02:25.223719 18860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:25.223827 18860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:25.223924 18860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:02:25.224009 18860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:02:25.233693 18860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:44403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:33771,127.4.82.189:45051,127.4.82.188:32775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.189.keytab
--principal=kudu/127.4.82.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.189:45051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.82.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:25.234007 18852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } attempt: 4
I20260809 01:02:25.234989 18860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:25.235230 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.234650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:45051 (local address 127.0.0.1:46980)
0809 01:02:25.234969 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.235052 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260809 01:02:25.236260 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.235933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:45051 (local address 127.0.0.1:46994)
0809 01:02:25.236065 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.236148 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 01:02:25.236439 18852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:45051: Network error: Client connection negotiation failed: client connection to 127.4.82.189:45051: connect: Connection refused (error 111)
I20260809 01:02:25.236682 18860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:25.248219 18867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:25.248332 18866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:25.248319 18869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:25.249684 18860 server_base.cc:1034] running on GCE node
I20260809 01:02:25.250649 18860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:25.252034 18860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:25.253268 18860 hybrid_clock.cc:648] HybridClock initialized: now 1786237345253224 us; error 31 us; skew 500 ppm
I20260809 01:02:25.253625 18860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:25 dist-test-slave-07c2 krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:25.262907 18860 init.cc:377] Logged in from keytab as kudu/127.4.82.189@KRBTEST.COM (short username kudu)
I20260809 01:02:25.264364 18860 webserver.cc:460] Webserver started at http://127.4.82.189:44841/ using document root <none> and password file <none>
I20260809 01:02:25.265038 18860 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:25.265178 18860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:25.265470 18860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:02:25.268492 18860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/data/instance:
uuid: "f84831a742274c17b42150a7440e3e2c"
format_stamp: "Formatted at 2026-08-09 01:02:25 on dist-test-slave-07c2"
server_key: "adbf8e02ebccfbd9749311a6a44c0eca"
server_key_iv: "03e096ce9d91d8cce785a4e675fc667c"
server_key_version: "encryptionkey@0"
I20260809 01:02:25.269333 18860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/wal/instance:
uuid: "f84831a742274c17b42150a7440e3e2c"
format_stamp: "Formatted at 2026-08-09 01:02:25 on dist-test-slave-07c2"
server_key: "adbf8e02ebccfbd9749311a6a44c0eca"
server_key_iv: "03e096ce9d91d8cce785a4e675fc667c"
server_key_version: "encryptionkey@0"
I20260809 01:02:25.274433 18860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 01:02:25.278183 18876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:25.279412 18860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:02:25.279614 18860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/wal
uuid: "f84831a742274c17b42150a7440e3e2c"
format_stamp: "Formatted at 2026-08-09 01:02:25 on dist-test-slave-07c2"
server_key: "adbf8e02ebccfbd9749311a6a44c0eca"
server_key_iv: "03e096ce9d91d8cce785a4e675fc667c"
server_key_version: "encryptionkey@0"
I20260809 01:02:25.279814 18860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:25.301584 18860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:25.305743 18860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:25.306025 18860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:25.323205 18860 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.189:45051
I20260809 01:02:25.323243 18929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.189:45051 every 8 connection(s)
I20260809 01:02:25.324558 18860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/data/info.pb
I20260809 01:02:25.327991 18930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:25.334139 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 18860
I20260809 01:02:25.334342 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-1/wal/instance
I20260809 01:02:25.334792 4426 external_mini_cluster.cc:1428] Setting key 8795a428c1e6d1f35eb93b8c8e6624e0
I20260809 01:02:25.334280 18930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:02:25 dist-test-slave-07c2 krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.189@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:25.354408 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.336980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48518 (local address 127.4.82.190:33771)
0809 01:02:25.337155 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:25.337159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:25.338360 (+ 1201us) server_negotiation.cc:407] Connection header received
0809 01:02:25.340148 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:25.340154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:25.340244 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:25.340350 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:25.341782 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.342458 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.344038 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.344450 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.344586 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.348050 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:25.348055 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:25.348058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:25.348088 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:25.350990 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.352108 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.352114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.352117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.352258 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.352697 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.352702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.352706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.352985 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:25.353156 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:25.353851 (+ 695us) server_negotiation.cc:299] Negotiation successful
0809 01:02:25.354089 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 01:02:25.355434 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.337461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33771 (local address 127.0.0.1:48518)
0809 01:02:25.338193 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.338241 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:25.339894 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:25.340604 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:25.340612 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:25.341094 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:25.341604 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.341619 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.342624 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.342628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.343879 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.343890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.344676 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.344683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.344928 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.345534 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:25.345559 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:25.347828 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:25.351157 (+ 3329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.351182 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.351198 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.351965 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.352388 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.352392 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.352396 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.352572 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.353124 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:25.353132 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:25.353475 (+ 343us) client_negotiation.cc:769] Sending connection context
0809 01:02:25.353736 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 01:02:25.354182 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":345,"threads_started":1}
I20260809 01:02:25.357340 18930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.82.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.188@KRBTEST.COM" created.
Aug 09 01:02:25 dist-test-slave-07c2 krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.189@KRBTEST.COM for kudu/127.4.82.189@KRBTEST.COM
I20260809 01:02:25.372529 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.358382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:45051 (local address 127.0.0.1:47008)
0809 01:02:25.358716 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.358737 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:25.358871 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:25.359787 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:25.359790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:25.360184 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:25.360327 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.360335 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.361727 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.361731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.362704 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.362722 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.363733 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.363744 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.364005 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.364583 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:25.364611 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:25.366966 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:25.370140 (+ 3174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.370145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.370149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.370482 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.370924 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.370927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.370929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.371070 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.371839 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:25.371842 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:25.371961 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 01:02:25.372101 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 01:02:25.372291 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260809 01:02:25.373071 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.358382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47008 (local address 127.4.82.189:45051)
0809 01:02:25.359228 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:25.359237 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:25.359273 (+ 36us) server_negotiation.cc:407] Connection header received
0809 01:02:25.359356 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:25.359366 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:25.359593 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:25.359758 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:25.360477 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.361586 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.362851 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.363574 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.363682 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.367168 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:25.367179 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:25.367190 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:25.367221 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:25.369984 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.370625 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.370632 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.370634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.370773 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.371178 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.371184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.371189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.371706 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:25.371932 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:25.372296 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 01:02:25.372564 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":475,"threads_started":1}
I20260809 01:02:25.375247 18930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:02:25.376413 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.376010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52030)
0809 01:02:25.376173 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.376290 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 01:02:25.376669 18877 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
I20260809 01:02:25.378166 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.377862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52046)
0809 01:02:25.377995 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.378073 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 01:02:25.378376 18930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:32775: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.188.keytab.
Entry for principal kudu/127.4.82.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.188.keytab.
I20260809 01:02:25.392709 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.188:32775
--webserver_interface=127.4.82.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.188.keytab
--principal=kudu/127.4.82.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:44403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:33771,127.4.82.189:45051,127.4.82.188:32775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:02:25.427657 18930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } attempt: 1
I20260809 01:02:25.428791 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.428314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52058)
0809 01:02:25.428505 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.428644 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 01:02:25.429780 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.429445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52062)
0809 01:02:25.429612 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.429676 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 01:02:25.429972 18930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:32775: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
I20260809 01:02:25.498155 18930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } attempt: 2
I20260809 01:02:25.499413 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.498890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52072)
0809 01:02:25.499087 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.499179 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 01:02:25.500418 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.500015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52078)
0809 01:02:25.500176 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.500272 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 01:02:25.500619 18930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:32775: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
I20260809 01:02:25.535730 18852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } attempt: 5
Aug 09 01:02:25 dist-test-slave-07c2 krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237344, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for kudu/127.4.82.189@KRBTEST.COM
I20260809 01:02:25.551416 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.536686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47010 (local address 127.4.82.189:45051)
0809 01:02:25.536873 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:25.536877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:25.536897 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:02:25.537100 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:25.537104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:25.537183 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:25.537269 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:25.538312 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.539047 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.540907 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.541362 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.541465 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.545648 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:25.545653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:25.545656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:25.545687 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:25.548309 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.549259 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.549263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.549265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.549387 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.549888 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.549891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.549893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.550163 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:25.550287 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:25.550861 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 01:02:25.551109 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 01:02:25.551649 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.536521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:45051 (local address 127.0.0.1:47010)
0809 01:02:25.536781 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.536807 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:25.536961 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:25.537356 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:25.537362 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:25.537888 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:25.538136 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.538149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.539191 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.539197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.540717 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.540739 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.541535 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.541541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.541712 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.542475 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:25.542508 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:25.545460 (+ 2952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:25.548462 (+ 3002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.548470 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.548476 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.549134 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.549514 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.549521 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.549525 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.549735 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.550300 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:25.550308 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:25.550536 (+ 228us) client_negotiation.cc:769] Sending connection context
0809 01:02:25.550768 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 01:02:25.551158 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 01:02:25.552724 18852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:02:25.553841 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.553336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52092)
0809 01:02:25.553543 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.553653 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 01:02:25.554872 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.554508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52102)
0809 01:02:25.554658 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.554751 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 01:02:25.555095 18852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:32775: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
I20260809 01:02:25.591822 18930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } attempt: 3
I20260809 01:02:25.593045 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.592594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52110)
0809 01:02:25.592801 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.592891 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 01:02:25.594177 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.593785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52120)
0809 01:02:25.593993 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.594065 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260809 01:02:25.594462 18930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:32775: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
I20260809 01:02:25.606412 18852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } attempt: 1
I20260809 01:02:25.607950 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.607409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52124)
0809 01:02:25.607625 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.607746 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 01:02:25.609236 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.608844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52140)
0809 01:02:25.609011 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.609108 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 01:02:25.609481 18852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:32775: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
I20260809 01:02:25.677731 18852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } attempt: 2
I20260809 01:02:25.679069 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.678433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52144)
0809 01:02:25.678724 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.678846 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 01:02:25.680171 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.679779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52150)
0809 01:02:25.679930 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.680022 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 01:02:25.680384 18852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:32775: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
W20260809 01:02:25.729857 18938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:25.730173 18938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:25.730345 18938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:25.737311 18938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:02:25.737437 18938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:25.737519 18938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:25.737627 18938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:02:25.737694 18938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:02:25.737823 18930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } attempt: 4
I20260809 01:02:25.738945 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.738489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52154)
0809 01:02:25.738704 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.738792 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 01:02:25.739959 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.739638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52170)
0809 01:02:25.739785 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.739861 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 01:02:25.740211 18930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:32775: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
I20260809 01:02:25.747905 18938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:44403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:33771,127.4.82.189:45051,127.4.82.188:32775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.188.keytab
--principal=kudu/127.4.82.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.188:32775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.82.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:25.749130 18938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:25.750394 18938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:25.761904 18944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:25.761968 18943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:25.762008 18946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:25.763489 18938 server_base.cc:1034] running on GCE node
I20260809 01:02:25.764297 18938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:25.765540 18938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:25.766776 18938 hybrid_clock.cc:648] HybridClock initialized: now 1786237345766722 us; error 43 us; skew 500 ppm
I20260809 01:02:25.767109 18938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:25 dist-test-slave-07c2 krb5kdc[18766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:25.770134 18938 init.cc:377] Logged in from keytab as kudu/127.4.82.188@KRBTEST.COM (short username kudu)
I20260809 01:02:25.771564 18938 webserver.cc:460] Webserver started at http://127.4.82.188:40437/ using document root <none> and password file <none>
I20260809 01:02:25.772197 18938 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:25.772343 18938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:25.772647 18938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:02:25.775735 18938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/data/instance:
uuid: "34e286960bde43d3a9b84161c50df82c"
format_stamp: "Formatted at 2026-08-09 01:02:25 on dist-test-slave-07c2"
server_key: "75800cfca4b46a861a87da0270c3d2c6"
server_key_iv: "90e0115dd439f8d41437f080870e4a29"
server_key_version: "encryptionkey@0"
I20260809 01:02:25.776620 18938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/wal/instance:
uuid: "34e286960bde43d3a9b84161c50df82c"
format_stamp: "Formatted at 2026-08-09 01:02:25 on dist-test-slave-07c2"
server_key: "75800cfca4b46a861a87da0270c3d2c6"
server_key_iv: "90e0115dd439f8d41437f080870e4a29"
server_key_version: "encryptionkey@0"
I20260809 01:02:25.782035 18938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 01:02:25.785619 18953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:25.786593 18852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } attempt: 3
I20260809 01:02:25.786945 18938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 01:02:25.787117 18938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/wal
uuid: "34e286960bde43d3a9b84161c50df82c"
format_stamp: "Formatted at 2026-08-09 01:02:25 on dist-test-slave-07c2"
server_key: "75800cfca4b46a861a87da0270c3d2c6"
server_key_iv: "90e0115dd439f8d41437f080870e4a29"
server_key_version: "encryptionkey@0"
I20260809 01:02:25.787366 18938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:25.787755 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.787212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52176)
0809 01:02:25.787454 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.787532 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260809 01:02:25.788758 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.788393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52190)
0809 01:02:25.788578 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.788650 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260809 01:02:25.789006 18852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:32775: Network error: Client connection negotiation failed: client connection to 127.4.82.188:32775: connect: Connection refused (error 111)
I20260809 01:02:25.809831 18938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:25.813874 18938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:25.814146 18938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:25.833529 18938 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.188:32775
I20260809 01:02:25.833554 19008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.188:32775 every 8 connection(s)
I20260809 01:02:25.834801 18938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/data/info.pb
I20260809 01:02:25.835608 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 18938
I20260809 01:02:25.835843 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786237144174446-4426-0/minicluster-data/master-2/wal/instance
I20260809 01:02:25.836380 4426 external_mini_cluster.cc:1428] Setting key 5faa26d68e9e40ac30adf0285ae9f8ec
I20260809 01:02:25.839005 19009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:25.847164 19009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:02:25 dist-test-slave-07c2 krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.188@KRBTEST.COM
Aug 09 01:02:25 dist-test-slave-07c2 krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.188@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:25.860234 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.838150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52196 (local address 127.4.82.188:32775)
0809 01:02:25.838901 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:25.838912 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:25.838949 (+ 37us) server_negotiation.cc:407] Connection header received
0809 01:02:25.840620 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:25.840639 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:25.840956 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:25.841288 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:25.843114 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.844517 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.846454 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.847075 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.847198 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.852197 (+ 4999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:25.852209 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:25.852220 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:25.852269 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:25.855927 (+ 3658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.856784 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.856795 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.856805 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.857029 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.857439 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.857447 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.857455 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.858150 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:25.858386 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:25.858902 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 01:02:25.859278 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":354,"threads_started":1}
I20260809 01:02:25.865852 19013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.850114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:33771 (local address 127.0.0.1:48524)
0809 01:02:25.850883 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.850928 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:25.851118 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:25.851577 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:25.851582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:25.852239 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:25.852646 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.852658 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.853663 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.853669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.855431 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.855445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.856215 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.856220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.856344 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.856920 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:25.856943 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:25.859611 (+ 2668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:25.862922 (+ 3311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.862934 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.862943 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.863612 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.864003 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.864009 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.864013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.864197 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.864854 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:25.864860 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:25.864982 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 01:02:25.865155 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 01:02:25.865448 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":414,"threads_started":1}
I20260809 01:02:25.865916 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.850118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48524 (local address 127.4.82.190:33771)
0809 01:02:25.850322 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:25.850332 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:25.851010 (+ 678us) server_negotiation.cc:407] Connection header received
0809 01:02:25.851272 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:25.851279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:25.851388 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:25.851508 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:25.852804 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.853486 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.855573 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.856015 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.856170 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.859806 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:25.859811 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:25.859814 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:25.859846 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:25.862776 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.863751 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.863755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.863758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.863893 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.864346 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.864351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.864356 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.864712 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:25.864884 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:25.865277 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 01:02:25.865474 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 01:02:25.867075 19009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:02:25 dist-test-slave-07c2 krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
Aug 09 01:02:25 dist-test-slave-07c2 krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.188@KRBTEST.COM for kudu/127.4.82.189@KRBTEST.COM
I20260809 01:02:25.878328 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.863193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48532 (local address 127.4.82.190:33771)
0809 01:02:25.863875 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:25.863879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:25.863897 (+ 18us) server_negotiation.cc:407] Connection header received
0809 01:02:25.863935 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:25.863963 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:25.864031 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:25.864142 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:25.865594 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.866684 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.868283 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.868679 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.868808 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.872735 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:25.872740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:25.872743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:25.872771 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:25.875633 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.876418 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.876423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.876425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.876536 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.876909 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.876914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.876924 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.877171 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:25.877446 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:25.877746 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 01:02:25.877956 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":291,"threads_started":1}
I20260809 01:02:25.879644 19013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.867831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:45051 (local address 127.0.0.1:47012)
0809 01:02:25.868104 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.868118 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:25.868244 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:25.868572 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:25.868577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:25.868916 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:25.869040 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.869056 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.869993 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.870001 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.870839 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.870849 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.871593 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.871598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.871822 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.872322 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:25.872337 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:25.874309 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:25.877208 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.877215 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.877219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.877568 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.877954 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.877959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.877963 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.878152 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.878778 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:25.878782 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:25.878956 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 01:02:25.879121 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 01:02:25.879394 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184}
I20260809 01:02:25.880065 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.867832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47012 (local address 127.4.82.189:45051)
0809 01:02:25.867993 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:25.867998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:25.868193 (+ 195us) server_negotiation.cc:407] Connection header received
0809 01:02:25.868364 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:25.868370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:25.868440 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:25.868542 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:25.869194 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.869859 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.870976 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.871445 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.871585 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.874498 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:25.874503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:25.874506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:25.874536 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:25.877068 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.877691 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.877697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.877701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.877841 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.878284 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.878291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.878294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.878627 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:25.878787 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:25.879234 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 01:02:25.879506 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 01:02:25.880998 19009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:02:25 dist-test-slave-07c2 krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.188@KRBTEST.COM for kudu/127.4.82.188@KRBTEST.COM
Aug 09 01:02:25 dist-test-slave-07c2 krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.189@KRBTEST.COM
I20260809 01:02:25.894750 19013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.881679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52210)
0809 01:02:25.881818 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.881836 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:25.881923 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:25.882350 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:25.882355 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:25.882825 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:25.883017 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.883027 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.884277 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.884281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.885397 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.885411 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.886373 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.886379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.886522 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.887004 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:25.887020 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:25.889060 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:25.892340 (+ 3280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.892346 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.892350 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.892726 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.893129 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.893132 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.893135 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.893265 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.893863 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:25.893868 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:25.894032 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 01:02:25.894212 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 01:02:25.894426 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 01:02:25.895053 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.881902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52210 (local address 127.4.82.188:32775)
0809 01:02:25.882067 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:25.882073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:25.882101 (+ 28us) server_negotiation.cc:407] Connection header received
0809 01:02:25.882150 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:25.882154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:25.882219 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:25.882314 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:25.883147 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.884162 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.885530 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.886213 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.886343 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.889260 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:25.889264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:25.889267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:25.889294 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:25.892217 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.892841 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.892846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.892849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.893019 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.893408 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.893412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.893413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.893720 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:25.893870 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:25.894330 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 01:02:25.894624 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":58}
I20260809 01:02:25.896099 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.880180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47024 (local address 127.4.82.189:45051)
0809 01:02:25.880575 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:25.880579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:25.880591 (+ 12us) server_negotiation.cc:407] Connection header received
0809 01:02:25.880621 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:25.880624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:25.880692 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:25.880788 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:25.882336 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.883408 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.884891 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.885549 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.885644 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.890037 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:25.890042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:25.890045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:25.890072 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:25.892893 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.893585 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.893590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.893593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.893748 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.894158 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.894163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.894166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.894503 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:25.894788 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:25.895677 (+ 889us) server_negotiation.cc:299] Negotiation successful
0809 01:02:25.895854 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":321}
I20260809 01:02:25.900274 19009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c: Bootstrap starting.
I20260809 01:02:25.900672 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:02:25.903216 19009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:25.904373 19009 log.cc:824] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:25.907637 19009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c: No bootstrap required, opened a new log
I20260809 01:02:25.918821 19009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } }
I20260809 01:02:25.919396 19009 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:25.919576 19009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e286960bde43d3a9b84161c50df82c, State: Initialized, Role: FOLLOWER
I20260809 01:02:25.920284 19009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } }
I20260809 01:02:25.925253 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.905501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48544 (local address 127.4.82.190:33771)
0809 01:02:25.905696 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:25.905703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:25.906269 (+ 566us) server_negotiation.cc:407] Connection header received
0809 01:02:25.906466 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:25.906470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:25.906548 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:25.906661 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:25.909223 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.910278 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.912435 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.913039 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.913155 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.917067 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:25.917073 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:25.917078 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:25.917118 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:25.920687 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.921694 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.921703 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.921709 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.921886 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.922409 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.922417 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.922423 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.922858 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:25.923142 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:25.924238 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0809 01:02:25.924525 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 01:02:25.926363 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.905986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47032 (local address 127.4.82.189:45051)
0809 01:02:25.906127 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:25.906133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:25.907337 (+ 1204us) server_negotiation.cc:407] Connection header received
0809 01:02:25.908203 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:25.908209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:25.908280 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:25.908344 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:25.910737 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.911898 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.914220 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.914889 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.914978 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.919073 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:25.919081 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:25.919088 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:25.919128 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:25.922867 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.923851 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.923857 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.923863 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.924015 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.924498 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.924506 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.924510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.924896 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:25.925151 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:25.925755 (+ 604us) server_negotiation.cc:299] Negotiation successful
0809 01:02:25.925980 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 01:02:25.926900 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.907216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52226 (local address 127.4.82.188:32775)
0809 01:02:25.907415 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:25.907423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:25.908120 (+ 697us) server_negotiation.cc:407] Connection header received
0809 01:02:25.908351 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:25.908356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:25.908418 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:25.908482 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:25.911060 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.912149 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.914257 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.914979 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.915061 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.919249 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:25.919256 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:25.919260 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:25.919292 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:25.923154 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.924253 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.924259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.924265 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.924413 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.925106 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.925111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.925114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.925521 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:25.925763 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:25.926301 (+ 538us) server_negotiation.cc:299] Negotiation successful
0809 01:02:25.926539 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 01:02:25.927430 19023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } } }
I20260809 01:02:25.927740 19023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:02:25.929252 19009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [sys.catalog]: configured and running, proceeding with master startup.
W20260809 01:02:25.941576 19034 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 01:02:25.941694 19034 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 01:02:25.938773 18938 master_runner.cc:186] Error getting master registration for 127.4.82.190:33771: OK, instance_id { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" instance_seqno: 1786237344822572 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 01:02:25.947343 18938 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:02:25.966288 18852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } attempt: 4
Aug 09 01:02:25 dist-test-slave-07c2 krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237344, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for kudu/127.4.82.188@KRBTEST.COM
I20260809 01:02:25.981905 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.966915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52242 (local address 127.4.82.188:32775)
0809 01:02:25.967054 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:25.967062 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:25.967084 (+ 22us) server_negotiation.cc:407] Connection header received
0809 01:02:25.967276 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:25.967302 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:25.967396 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:25.967498 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:25.968435 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.969573 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.971401 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.971925 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.972020 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.975631 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:25.975638 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:25.975642 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:25.975677 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:25.979099 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.979933 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.979938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.979941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.980116 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:25.980495 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:25.980500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:25.980505 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:25.980782 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:25.980932 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:25.981324 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 01:02:25.981523 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 01:02:25.981921 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:25.966840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52242)
0809 01:02:25.967023 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:25.967042 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:25.967154 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:25.967526 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:25.967530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:25.968063 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:25.968270 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.968282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.969709 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.969715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.971208 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:25.971224 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:25.972083 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:25.972090 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:25.972265 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:25.972868 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:25.972894 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:25.975449 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:25.979266 (+ 3817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.979271 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.979275 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.979800 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.980233 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:25.980240 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:25.980243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:25.980378 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:25.980902 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:25.980907 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:25.981070 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 01:02:25.981216 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 01:02:25.981516 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 01:02:25.986410 18852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b: Bootstrap starting.
I20260809 01:02:25.989494 18852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:25.990523 18852 log.cc:824] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:25.993955 18852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b: No bootstrap required, opened a new log
I20260809 01:02:26.001072 18852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } }
I20260809 01:02:26.001405 18852 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:26.001546 18852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fceeb58d148444bb0218eea82f0451b, State: Initialized, Role: FOLLOWER
I20260809 01:02:26.002210 18852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } }
I20260809 01:02:26.003803 19037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } } }
I20260809 01:02:26.004245 19037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:02:26.004902 18852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [sys.catalog]: configured and running, proceeding with master startup.
W20260809 01:02:26.018224 19048 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 01:02:26.018416 19048 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 01:02:26.014021 18780 master_runner.cc:186] Error getting master registration for 127.4.82.189:45051: OK, instance_id { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" instance_seqno: 1786237345318302 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 01:02:26.022737 18780 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:02:26.039424 18930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } attempt: 5
Aug 09 01:02:26 dist-test-slave-07c2 krb5kdc[18766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.189@KRBTEST.COM for kudu/127.4.82.188@KRBTEST.COM
I20260809 01:02:26.049729 19023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:02:26.050040 19023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } }
I20260809 01:02:26.051815 19023 leader_election.cc:288] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fceeb58d148444bb0218eea82f0451b (127.4.82.190:33771), f84831a742274c17b42150a7440e3e2c (127.4.82.189:45051)
I20260809 01:02:26.052896 18824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34e286960bde43d3a9b84161c50df82c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fceeb58d148444bb0218eea82f0451b" is_pre_election: true
I20260809 01:02:26.053472 18824 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e286960bde43d3a9b84161c50df82c in term 0.
I20260809 01:02:26.054272 18956 leader_election.cc:302] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e286960bde43d3a9b84161c50df82c, 8fceeb58d148444bb0218eea82f0451b; no voters:
I20260809 01:02:26.055507 19023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:02:26.055588 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:26.039997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:32775 (local address 127.0.0.1:52258)
0809 01:02:26.040257 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:26.040276 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:26.040393 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:26.040785 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:26.040790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:26.041223 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:26.041376 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:26.041385 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:26.043314 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:26.043319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:26.044652 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:26.044666 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:26.045415 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:26.045420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:26.045631 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:26.046248 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:26.046264 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:26.048257 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:26.052566 (+ 4309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:26.052570 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:26.052574 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:26.053104 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:26.053577 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:26.053584 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:26.053589 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:26.053762 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:26.054530 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:26.054534 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:26.054699 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 01:02:26.054849 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 01:02:26.055133 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260809 01:02:26.055677 19023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:02:26.055790 19023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:26.056612 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:26.040091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52258 (local address 127.4.82.188:32775)
0809 01:02:26.040258 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:26.040266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:26.040326 (+ 60us) server_negotiation.cc:407] Connection header received
0809 01:02:26.040587 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:26.040591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:26.040649 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:26.040757 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:26.041518 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:26.042626 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:26.044796 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:26.045268 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:26.045395 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:26.048442 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:26.048447 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:26.048450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:26.048478 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:26.052406 (+ 3928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:26.053272 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:26.053281 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:26.053288 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:26.053428 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:26.053898 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:26.053902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:26.053905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:26.054307 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:26.054463 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:26.055939 (+ 1476us) server_negotiation.cc:299] Negotiation successful
0809 01:02:26.056205 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 01:02:26.062099 18930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c: Bootstrap starting.
W20260809 01:02:26.062114 18802 tablet.cc:2369] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:02:26.064431 19023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } }
W20260809 01:02:26.066195 18959 tablet.cc:2369] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:02:26.066390 18930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:26.067770 18930 log.cc:824] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:26.068219 18824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34e286960bde43d3a9b84161c50df82c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fceeb58d148444bb0218eea82f0451b"
I20260809 01:02:26.068491 18824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:26.075623 18824 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e286960bde43d3a9b84161c50df82c in term 1.
I20260809 01:02:26.076295 18956 leader_election.cc:302] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e286960bde43d3a9b84161c50df82c, 8fceeb58d148444bb0218eea82f0451b; no voters:
I20260809 01:02:26.077286 18930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c: No bootstrap required, opened a new log
I20260809 01:02:26.085075 19023 leader_election.cc:288] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [CANDIDATE]: Term 1 election: Requested vote from peers 8fceeb58d148444bb0218eea82f0451b (127.4.82.190:33771), f84831a742274c17b42150a7440e3e2c (127.4.82.189:45051)
I20260809 01:02:26.085815 19023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:02:26.086432 19023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [term 1 LEADER]: Becoming Leader. State: Replica: 34e286960bde43d3a9b84161c50df82c, State: Running, Role: LEADER
I20260809 01:02:26.088528 19023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } }
I20260809 01:02:26.089507 18930 raft_consensus.cc:348] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } }
I20260809 01:02:26.089908 18930 raft_consensus.cc:374] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:26.090030 18930 raft_consensus.cc:729] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84831a742274c17b42150a7440e3e2c, State: Initialized, Role: FOLLOWER
I20260809 01:02:26.090817 18930 consensus_queue.cc:260] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } }
I20260809 01:02:26.097934 19052 sys_catalog.cc:455] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } } }
I20260809 01:02:26.098323 19052 sys_catalog.cc:458] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:02:26.100219 18903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34e286960bde43d3a9b84161c50df82c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84831a742274c17b42150a7440e3e2c"
I20260809 01:02:26.100660 18903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:26.100943 18904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34e286960bde43d3a9b84161c50df82c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84831a742274c17b42150a7440e3e2c" is_pre_election: true
I20260809 01:02:26.104310 18903 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e286960bde43d3a9b84161c50df82c in term 1.
I20260809 01:02:26.104681 18930 sys_catalog.cc:564] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:02:26.113978 19051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34e286960bde43d3a9b84161c50df82c. Latest consensus state: current_term: 1 leader_uuid: "34e286960bde43d3a9b84161c50df82c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } } }
I20260809 01:02:26.114508 19051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:26.123145 19056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:02:26.126101 19056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:02:26.147403 18903 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [term 1 FOLLOWER]: Refusing update from remote peer 34e286960bde43d3a9b84161c50df82c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:02:26.148406 19051 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:02:26.159860 18824 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [term 1 FOLLOWER]: Refusing update from remote peer 34e286960bde43d3a9b84161c50df82c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:02:26.160818 19051 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:02:26.182256 19037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34e286960bde43d3a9b84161c50df82c. Latest consensus state: current_term: 1 leader_uuid: "34e286960bde43d3a9b84161c50df82c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } } }
I20260809 01:02:26.182595 19037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:02:26.186491 19060 mvcc.cc:205] Tried to move back new op lower bound from 7316428169805824000 to 7316428169602887680. Current Snapshot: MvccSnapshot[applied={T|T < 7316428169805824000}]
I20260809 01:02:26.194551 19052 sys_catalog.cc:455] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34e286960bde43d3a9b84161c50df82c. Latest consensus state: current_term: 1 leader_uuid: "34e286960bde43d3a9b84161c50df82c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } } }
I20260809 01:02:26.194952 19052 sys_catalog.cc:458] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:02:26.195621 19037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34e286960bde43d3a9b84161c50df82c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } } }
I20260809 01:02:26.195932 19037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:02:26.196792 19067 mvcc.cc:205] Tried to move back new op lower bound from 7316428169805824000 to 7316428169602887680. Current Snapshot: MvccSnapshot[applied={T|T < 7316428169805824000}]
I20260809 01:02:26.200335 19023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34e286960bde43d3a9b84161c50df82c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } } }
I20260809 01:02:26.200745 19023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:26.205680 19065 mvcc.cc:205] Tried to move back new op lower bound from 7316428169805824000 to 7316428169602887680. Current Snapshot: MvccSnapshot[applied={T|T < 7316428169805824000}]
I20260809 01:02:26.207068 19052 sys_catalog.cc:455] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34e286960bde43d3a9b84161c50df82c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } } }
I20260809 01:02:26.207401 19052 sys_catalog.cc:458] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:02:26.219071 19056 catalog_manager.cc:1347] Generated new cluster ID: 6a6faaa5c50e40fe978cdd5a5288f4e1
I20260809 01:02:26.219213 19056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260809 01:02:26.224915 19073 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 01:02:26.225049 19073 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 01:02:26.231814 19051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34e286960bde43d3a9b84161c50df82c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fceeb58d148444bb0218eea82f0451b" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 33771 } } peers { permanent_uuid: "f84831a742274c17b42150a7440e3e2c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 45051 } } peers { permanent_uuid: "34e286960bde43d3a9b84161c50df82c" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 32775 } } }
I20260809 01:02:26.232079 19051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:26.249114 19056 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:02:26.251039 19056 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:02:26.270942 19056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34e286960bde43d3a9b84161c50df82c: Generated new TSK 0
I20260809 01:02:26.272156 19056 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:02:27.020913 19048 catalog_manager.cc:1259] Loaded cluster ID: 6a6faaa5c50e40fe978cdd5a5288f4e1
I20260809 01:02:27.021014 19048 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b: loading cluster ID for follower catalog manager: success
I20260809 01:02:27.023352 19048 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b: acquiring CA information for follower catalog manager: success
I20260809 01:02:27.025108 19048 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8fceeb58d148444bb0218eea82f0451b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 01:02:27.226842 19073 catalog_manager.cc:1259] Loaded cluster ID: 6a6faaa5c50e40fe978cdd5a5288f4e1
I20260809 01:02:27.226946 19073 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c: loading cluster ID for follower catalog manager: success
I20260809 01:02:27.229490 19073 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c: acquiring CA information for follower catalog manager: success
I20260809 01:02:27.231405 19073 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f84831a742274c17b42150a7440e3e2c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 01:02:27.424942 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 18780
I20260809 01:02:27.426357 18846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:27.535408 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 18780
I20260809 01:02:27.560045 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 18860
I20260809 01:02:27.561388 18924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:27.670543 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 18860
I20260809 01:02:27.693517 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 18938
I20260809 01:02:27.694567 19004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:27.803925 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 18938
2026-08-09T01:02:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4816 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:02:27 dist-test-slave-07c2 krb5kdc[19089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:02:27 dist-test-slave-07c2 krb5kdc[19089](info): set up 2 sockets
Aug 09 01:02:27 dist-test-slave-07c2 krb5kdc[19089](info): commencing operation
krb5kdc: starting...
W20260809 01:02:28.942862 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:02:29 dist-test-slave-07c2 krb5kdc[19089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:02:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:02:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:02:29.205430 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:40485
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:40485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:29.542872 19104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:29.543164 19104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:29.543365 19104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:29.550661 19104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:02:29.550765 19104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:29.550868 19104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:29.550964 19104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:02:29.551040 19104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:02:29.561055 19104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:40485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:40485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:29.562389 19104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:29.563763 19104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:29.575786 19109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:29.575800 19110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:29.576999 19112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:29.577637 19104 server_base.cc:1034] running on GCE node
I20260809 01:02:29.578629 19104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:29.580396 19104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:29.581634 19104 hybrid_clock.cc:648] HybridClock initialized: now 1786237349581560 us; error 57 us; skew 500 ppm
I20260809 01:02:29.582033 19104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:29 dist-test-slave-07c2 krb5kdc[19089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237349, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:29.586083 19104 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:02:29.587654 19104 webserver.cc:460] Webserver started at http://127.4.82.190:40931/ using document root <none> and password file <none>
I20260809 01:02:29.588397 19104 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:29.588531 19104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:29.588887 19104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:02:29.591977 19104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "2a378c9b255948759a163045e2c4a6ef"
format_stamp: "Formatted at 2026-08-09 01:02:29 on dist-test-slave-07c2"
server_key: "1676fd3592bf5f07bfe6f6365a7edf97"
server_key_iv: "fe52d0ac5ad3934de02e34c20da06de1"
server_key_version: "encryptionkey@0"
I20260809 01:02:29.592762 19104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "2a378c9b255948759a163045e2c4a6ef"
format_stamp: "Formatted at 2026-08-09 01:02:29 on dist-test-slave-07c2"
server_key: "1676fd3592bf5f07bfe6f6365a7edf97"
server_key_iv: "fe52d0ac5ad3934de02e34c20da06de1"
server_key_version: "encryptionkey@0"
I20260809 01:02:29.598048 19104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 01:02:29.601471 19119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:29.602680 19104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 01:02:29.602867 19104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "2a378c9b255948759a163045e2c4a6ef"
format_stamp: "Formatted at 2026-08-09 01:02:29 on dist-test-slave-07c2"
server_key: "1676fd3592bf5f07bfe6f6365a7edf97"
server_key_iv: "fe52d0ac5ad3934de02e34c20da06de1"
server_key_version: "encryptionkey@0"
I20260809 01:02:29.603068 19104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:29.623827 19104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:29.628154 19104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:29.628440 19104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:29.647214 19104 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:40485
I20260809 01:02:29.647253 19171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:40485 every 8 connection(s)
I20260809 01:02:29.648583 19104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:02:29.649755 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 19104
I20260809 01:02:29.649991 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:02:29.650678 4426 external_mini_cluster.cc:1428] Setting key 3c5cd71fb895752d95ccdc1c7054f5bd
I20260809 01:02:29.652932 19172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 01:02:29 dist-test-slave-07c2 krb5kdc[19089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:29.666250 19172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef: Bootstrap starting.
I20260809 01:02:29.669600 19172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:29.670647 19172 log.cc:824] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:29.674084 19172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef: No bootstrap required, opened a new log
I20260809 01:02:29.675132 19175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:29.652541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.4.82.190:40485)
0809 01:02:29.653408 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:29.653424 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:29.653473 (+ 49us) server_negotiation.cc:407] Connection header received
0809 01:02:29.655805 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:29.655831 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:29.656340 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:29.656764 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:29.658169 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:29.660235 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:29.661609 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:29.662654 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:29.662885 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:29.667133 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:29.667141 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:29.667157 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:29.667193 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:29.670375 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:29.671237 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:29.671250 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:29.671260 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:29.671489 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:29.671954 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:29.671957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:29.671959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:29.672690 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:29.673003 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:29.673376 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 01:02:29.673810 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":368,"threads_started":1}
I20260809 01:02:29.678928 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:02:29.682405 19172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a378c9b255948759a163045e2c4a6ef" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 40485 } }
I20260809 01:02:29.682761 19172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:29.682880 19172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a378c9b255948759a163045e2c4a6ef, State: Initialized, Role: FOLLOWER
I20260809 01:02:29.683502 19172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a378c9b255948759a163045e2c4a6ef" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 40485 } }
I20260809 01:02:29.683749 19172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:02:29.683914 19172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:02:29.684092 19172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:29.686197 19172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a378c9b255948759a163045e2c4a6ef" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 40485 } }
I20260809 01:02:29.686688 19172 leader_election.cc:302] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a378c9b255948759a163045e2c4a6ef; no voters:
I20260809 01:02:29.687269 19172 leader_election.cc:288] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:02:29.687471 19177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:02:29.688202 19177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [term 1 LEADER]: Becoming Leader. State: Replica: 2a378c9b255948759a163045e2c4a6ef, State: Running, Role: LEADER
I20260809 01:02:29.688719 19177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a378c9b255948759a163045e2c4a6ef" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 40485 } }
I20260809 01:02:29.689397 19172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:02:29.691574 19179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a378c9b255948759a163045e2c4a6ef. Latest consensus state: current_term: 1 leader_uuid: "2a378c9b255948759a163045e2c4a6ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a378c9b255948759a163045e2c4a6ef" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 40485 } } }
I20260809 01:02:29.691998 19179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:29.692734 19178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a378c9b255948759a163045e2c4a6ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a378c9b255948759a163045e2c4a6ef" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 40485 } } }
I20260809 01:02:29.693009 19178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:29.693638 19185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:02:29.698417 19185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:02:29.706348 19185 catalog_manager.cc:1347] Generated new cluster ID: 60ccbc8b67dc471c843f33c978846e48
I20260809 01:02:29.706468 19185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:02:29.719056 19185 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:02:29.720948 19185 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:02:29.728475 19185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a378c9b255948759a163045e2c4a6ef: Generated new TSK 0
I20260809 01:02:29.729538 19185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:02:29.739794 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 19104
I20260809 01:02:29.740998 19167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:29.829958 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 19104
2026-08-09T01:02:29Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2021 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:02:29 dist-test-slave-07c2 krb5kdc[19200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:02:29 dist-test-slave-07c2 krb5kdc[19200](info): set up 2 sockets
Aug 09 01:02:29 dist-test-slave-07c2 krb5kdc[19200](info): commencing operation
krb5kdc: starting...
W20260809 01:02:30.978868 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:02:31 dist-test-slave-07c2 krb5kdc[19200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:02:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:02:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:02:31.239239 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41471
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:41471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:31.571084 19214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:31.571434 19214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:31.571535 19214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:31.581343 19214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:02:31.581444 19214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:31.581489 19214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:31.581545 19214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:02:31.581597 19214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:02:31.591137 19214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:41471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:31.592367 19214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:31.593549 19214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:31.604899 19221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:31.604875 19222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:31.605060 19225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:31.606058 19214 server_base.cc:1034] running on GCE node
I20260809 01:02:31.606856 19214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:31.608184 19214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:31.609402 19214 hybrid_clock.cc:648] HybridClock initialized: now 1786237351609364 us; error 37 us; skew 500 ppm
I20260809 01:02:31.609730 19214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:31 dist-test-slave-07c2 krb5kdc[19200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237351, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:31.617568 19214 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:02:31.618983 19214 webserver.cc:460] Webserver started at http://127.4.82.190:45203/ using document root <none> and password file <none>
I20260809 01:02:31.619660 19214 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:31.619762 19214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:31.620002 19214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:02:31.623095 19214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "fb7a947c18db4082b29f796cedd5f916"
format_stamp: "Formatted at 2026-08-09 01:02:31 on dist-test-slave-07c2"
server_key: "df59c2784ffbac4579e1f3db0cb6e9da"
server_key_iv: "501739c14d8377e0c7556eaefb30757d"
server_key_version: "encryptionkey@0"
I20260809 01:02:31.623948 19214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "fb7a947c18db4082b29f796cedd5f916"
format_stamp: "Formatted at 2026-08-09 01:02:31 on dist-test-slave-07c2"
server_key: "df59c2784ffbac4579e1f3db0cb6e9da"
server_key_iv: "501739c14d8377e0c7556eaefb30757d"
server_key_version: "encryptionkey@0"
I20260809 01:02:31.629168 19214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 01:02:31.633107 19232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:31.634390 19214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 01:02:31.634565 19214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "fb7a947c18db4082b29f796cedd5f916"
format_stamp: "Formatted at 2026-08-09 01:02:31 on dist-test-slave-07c2"
server_key: "df59c2784ffbac4579e1f3db0cb6e9da"
server_key_iv: "501739c14d8377e0c7556eaefb30757d"
server_key_version: "encryptionkey@0"
I20260809 01:02:31.634732 19214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:31.681128 19214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:31.685415 19214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:31.685704 19214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:31.704591 19214 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:41471
I20260809 01:02:31.704628 19283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:41471 every 8 connection(s)
I20260809 01:02:31.705884 19214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:02:31.709439 19284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:31.715351 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 19214
I20260809 01:02:31.715566 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:02:31.716279 4426 external_mini_cluster.cc:1428] Setting key f573e85265d1866f53cbd9f1269cc3f0
I20260809 01:02:31.719990 19284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Bootstrap starting.
I20260809 01:02:31.723616 19284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:31.724620 19284 log.cc:824] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:31.727380 19284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: No bootstrap required, opened a new log
Aug 09 01:02:31 dist-test-slave-07c2 krb5kdc[19200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:31.735446 19284 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } }
I20260809 01:02:31.735910 19284 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:31.736025 19284 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb7a947c18db4082b29f796cedd5f916, State: Initialized, Role: FOLLOWER
I20260809 01:02:31.736632 19284 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } }
I20260809 01:02:31.736882 19284 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:02:31.737051 19284 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:02:31.737279 19284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:31.739275 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:31.718233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42070 (local address 127.4.82.190:41471)
0809 01:02:31.718979 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:31.718990 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:31.719029 (+ 39us) server_negotiation.cc:407] Connection header received
0809 01:02:31.720763 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:31.720783 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:31.721118 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:31.721450 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:31.723260 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:31.724653 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:31.726381 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:31.727017 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:31.727200 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:31.731707 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:31.731715 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:31.731732 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:31.731775 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:31.734827 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:31.735685 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:31.735692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:31.735697 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:31.735894 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:31.736367 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:31.736373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:31.736375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:31.736983 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:31.737325 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:31.737720 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 01:02:31.738138 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":355,"threads_started":1}
I20260809 01:02:31.739997 19284 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } }
I20260809 01:02:31.740469 19284 leader_election.cc:302] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb7a947c18db4082b29f796cedd5f916; no voters:
I20260809 01:02:31.741187 19284 leader_election.cc:288] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:02:31.741323 19289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:02:31.743643 19284 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:02:31.746803 19289 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 1 LEADER]: Becoming Leader. State: Replica: fb7a947c18db4082b29f796cedd5f916, State: Running, Role: LEADER
I20260809 01:02:31.747450 19289 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } }
I20260809 01:02:31.750671 19290 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb7a947c18db4082b29f796cedd5f916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } } }
I20260809 01:02:31.750698 19292 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb7a947c18db4082b29f796cedd5f916. Latest consensus state: current_term: 1 leader_uuid: "fb7a947c18db4082b29f796cedd5f916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } } }
I20260809 01:02:31.751240 19290 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:31.751273 19292 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: This master's current role is: LEADER
W20260809 01:02:31.755676 19304 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 01:02:31.755802 19304 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 01:02:31.757023 19305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:02:31.758625 19305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:02:31.766250 19305 catalog_manager.cc:1347] Generated new cluster ID: 919aede74558473eb07f04873c05e53a
I20260809 01:02:31.766366 19305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:02:31.775494 19305 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:02:31.776667 19305 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:02:31.784703 19305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Generated new TSK 0
I20260809 01:02:31.785418 19305 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 01:02:31.846622 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41471
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:32.181931 19311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:32.182201 19311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:32.182374 19311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:32.189697 19311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:32.189831 19311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:32.190017 19311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:02:32.199571 19311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:32.200980 19311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:32.202250 19311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:32.215531 19316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:32.215473 19319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:32.215507 19317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:32.216699 19311 server_base.cc:1034] running on GCE node
I20260809 01:02:32.217293 19311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:32.218202 19311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:32.219444 19311 hybrid_clock.cc:648] HybridClock initialized: now 1786237352219393 us; error 53 us; skew 500 ppm
I20260809 01:02:32.219767 19311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:32 dist-test-slave-07c2 krb5kdc[19200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237352, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:32.222919 19311 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:02:32.224305 19311 webserver.cc:460] Webserver started at http://127.4.82.129:37677/ using document root <none> and password file <none>
I20260809 01:02:32.224984 19311 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:32.225128 19311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:32.225454 19311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:02:32.228372 19311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "10a7ff25c6194e618d721e2597074ccb"
format_stamp: "Formatted at 2026-08-09 01:02:32 on dist-test-slave-07c2"
server_key: "bc0e5bad69ee34e73976ac362440aa69"
server_key_iv: "b617926b9e39cc1ee59ceb81ba4ff596"
server_key_version: "encryptionkey@0"
I20260809 01:02:32.229140 19311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "10a7ff25c6194e618d721e2597074ccb"
format_stamp: "Formatted at 2026-08-09 01:02:32 on dist-test-slave-07c2"
server_key: "bc0e5bad69ee34e73976ac362440aa69"
server_key_iv: "b617926b9e39cc1ee59ceb81ba4ff596"
server_key_version: "encryptionkey@0"
I20260809 01:02:32.234102 19311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 01:02:32.237339 19327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:32.238624 19311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 01:02:32.238801 19311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "10a7ff25c6194e618d721e2597074ccb"
format_stamp: "Formatted at 2026-08-09 01:02:32 on dist-test-slave-07c2"
server_key: "bc0e5bad69ee34e73976ac362440aa69"
server_key_iv: "b617926b9e39cc1ee59ceb81ba4ff596"
server_key_version: "encryptionkey@0"
I20260809 01:02:32.239048 19311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:32.255260 19311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:32.259451 19311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:32.259737 19311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:32.260633 19311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:32.261914 19311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:32.262017 19311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:32.262188 19311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:32.262267 19311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:32.296123 19311 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:35817
I20260809 01:02:32.296207 19445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:35817 every 8 connection(s)
I20260809 01:02:32.297396 19311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:02:32.304518 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 19311
I20260809 01:02:32.304759 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:02:32.305593 4426 external_mini_cluster.cc:1428] Setting key 9624718743c41ecd135c861c0e6a8043
Aug 09 01:02:32 dist-test-slave-07c2 krb5kdc[19200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237352, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:32.319049 19448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:32.299930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:46703 (local address 127.4.82.190:41471)
0809 01:02:32.300344 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:32.300349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:32.301337 (+ 988us) server_negotiation.cc:407] Connection header received
0809 01:02:32.303039 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:32.303044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:32.303109 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:32.303213 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:32.304739 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:32.305918 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:32.308074 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:32.308750 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:32.308892 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:32.312345 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:32.312354 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:32.312362 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:32.312407 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:32.316050 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:32.316785 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:32.316793 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:32.316800 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:32.316956 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:32.317367 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:32.317374 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:32.317380 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:32.317761 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:32.317923 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:32.318607 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 01:02:32.318785 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":208,"threads_started":1}
I20260809 01:02:32.320533 19449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:32.300475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:46703)
0809 01:02:32.301170 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:32.301211 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:32.302806 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:32.303480 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:32.303488 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:32.303987 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:32.304523 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:32.304539 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:32.306081 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:32.306085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:32.307929 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:32.307941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:32.308951 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:32.308955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:32.309146 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:32.309847 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:32.309869 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:32.312123 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:32.316205 (+ 4082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:32.316211 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:32.316230 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:32.316654 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:32.317061 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:32.317064 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:32.317066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:32.317251 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:32.317884 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:32.317890 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:32.318198 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 01:02:32.318436 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 01:02:32.319055 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":279,"threads_started":1}
I20260809 01:02:32.322239 19446 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41471
I20260809 01:02:32.322539 19446 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:32.323271 19446 heartbeater.cc:511] Master 127.4.82.190:41471 requested a full tablet report, sending...
I20260809 01:02:32.325666 19249 ts_manager.cc:194] Registered new tserver with Master: 10a7ff25c6194e618d721e2597074ccb (127.4.82.129:35817)
I20260809 01:02:32.327879 19249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:46703
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 01:02:32.364563 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41471
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:32.687245 19452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:32.687574 19452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:32.687757 19452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:32.694825 19452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:32.694976 19452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:32.695170 19452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:02:32.704726 19452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:32.705933 19452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:32.707191 19452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:32.719575 19457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:32.719631 19458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:32.720381 19460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:32.721050 19452 server_base.cc:1034] running on GCE node
I20260809 01:02:32.721611 19452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:32.722470 19452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:32.723690 19452 hybrid_clock.cc:648] HybridClock initialized: now 1786237352723606 us; error 96 us; skew 500 ppm
I20260809 01:02:32.723996 19452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:32 dist-test-slave-07c2 krb5kdc[19200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237352, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:32.727002 19452 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:02:32.728468 19452 webserver.cc:460] Webserver started at http://127.4.82.130:46269/ using document root <none> and password file <none>
I20260809 01:02:32.729128 19452 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:32.729251 19452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:32.729553 19452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:02:32.732371 19452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4"
format_stamp: "Formatted at 2026-08-09 01:02:32 on dist-test-slave-07c2"
server_key: "effc839db13e3fdde54dcf47d29f412d"
server_key_iv: "d75f7ebf24f547d2cb2e2695b838a0fa"
server_key_version: "encryptionkey@0"
I20260809 01:02:32.733093 19452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4"
format_stamp: "Formatted at 2026-08-09 01:02:32 on dist-test-slave-07c2"
server_key: "effc839db13e3fdde54dcf47d29f412d"
server_key_iv: "d75f7ebf24f547d2cb2e2695b838a0fa"
server_key_version: "encryptionkey@0"
I20260809 01:02:32.738071 19452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 01:02:32.741312 19467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:32.742614 19452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 01:02:32.742738 19452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4"
format_stamp: "Formatted at 2026-08-09 01:02:32 on dist-test-slave-07c2"
server_key: "effc839db13e3fdde54dcf47d29f412d"
server_key_iv: "d75f7ebf24f547d2cb2e2695b838a0fa"
server_key_version: "encryptionkey@0"
I20260809 01:02:32.742904 19452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:32.762171 19452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:32.766305 19452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:32.766556 19452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:32.767566 19452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:32.768965 19452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:32.769038 19452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:32.769119 19452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:32.769163 19452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:32.804461 19452 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:41017
I20260809 01:02:32.804514 19584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:41017 every 8 connection(s)
I20260809 01:02:32.805816 19452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:02:32.808820 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 19452
I20260809 01:02:32.809011 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:02:32.809676 4426 external_mini_cluster.cc:1428] Setting key c5d6a9b79b1415f7cf67e56df8b56b07
Aug 09 01:02:32 dist-test-slave-07c2 krb5kdc[19200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237352, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:32.826737 19448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:32.808451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:57425 (local address 127.4.82.190:41471)
0809 01:02:32.808655 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:32.808659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:32.810229 (+ 1570us) server_negotiation.cc:407] Connection header received
0809 01:02:32.812321 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:32.812326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:32.812421 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:32.812517 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:32.813967 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:32.814685 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:32.816335 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:32.816767 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:32.816901 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:32.820430 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:32.820440 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:32.820444 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:32.820478 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:32.823771 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:32.824476 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:32.824483 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:32.824489 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:32.824649 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:32.825064 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:32.825068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:32.825070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:32.825336 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:32.825496 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:32.826148 (+ 652us) server_negotiation.cc:299] Negotiation successful
0809 01:02:32.826378 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 01:02:32.827768 19587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:32.809152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:57425)
0809 01:02:32.810038 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:32.810094 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:32.812057 (+ 1963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:32.812765 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:32.812772 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:32.813267 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:32.813799 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:32.813815 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:32.814855 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:32.814859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:32.816187 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:32.816200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:32.816999 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:32.817006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:32.817228 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:32.817915 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:32.817937 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:32.820156 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:32.823902 (+ 3746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:32.823909 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:32.823920 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:32.824340 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:32.824756 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:32.824759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:32.824761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:32.824929 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:32.825476 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:32.825484 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:32.825785 (+ 301us) client_negotiation.cc:769] Sending connection context
0809 01:02:32.826050 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 01:02:32.826585 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":323,"threads_started":1}
I20260809 01:02:32.829169 19585 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41471
I20260809 01:02:32.829471 19585 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:32.830154 19585 heartbeater.cc:511] Master 127.4.82.190:41471 requested a full tablet report, sending...
I20260809 01:02:32.831676 19249 ts_manager.cc:194] Registered new tserver with Master: 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 (127.4.82.130:41017)
I20260809 01:02:32.832459 19249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:57425
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 01:02:32.870057 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41471
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:33.202180 19590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:33.202441 19590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:33.202601 19590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:33.209861 19590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:33.209986 19590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:33.210171 19590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:02:33.219846 19590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:41471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:33.221107 19590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:33.222400 19590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:33.235180 19596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:33.235219 19598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:33.235602 19595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:33.236502 19590 server_base.cc:1034] running on GCE node
I20260809 01:02:33.237104 19590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:33.238013 19590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:33.239243 19590 hybrid_clock.cc:648] HybridClock initialized: now 1786237353239188 us; error 41 us; skew 500 ppm
I20260809 01:02:33.239598 19590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:33 dist-test-slave-07c2 krb5kdc[19200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237353, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:33.242719 19590 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:02:33.244324 19590 webserver.cc:460] Webserver started at http://127.4.82.131:34771/ using document root <none> and password file <none>
I20260809 01:02:33.244987 19590 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:33.245141 19590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:33.245457 19590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:02:33.248453 19590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "da002cbbe93848e7a8188d112dc753c7"
format_stamp: "Formatted at 2026-08-09 01:02:33 on dist-test-slave-07c2"
server_key: "8a6f94006ebfeab36de78c031e3f4a62"
server_key_iv: "e2f296402b8b67d066a14628d8ed5f5b"
server_key_version: "encryptionkey@0"
I20260809 01:02:33.249200 19590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "da002cbbe93848e7a8188d112dc753c7"
format_stamp: "Formatted at 2026-08-09 01:02:33 on dist-test-slave-07c2"
server_key: "8a6f94006ebfeab36de78c031e3f4a62"
server_key_iv: "e2f296402b8b67d066a14628d8ed5f5b"
server_key_version: "encryptionkey@0"
I20260809 01:02:33.254325 19590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 01:02:33.257656 19605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:33.258903 19590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 01:02:33.259078 19590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "da002cbbe93848e7a8188d112dc753c7"
format_stamp: "Formatted at 2026-08-09 01:02:33 on dist-test-slave-07c2"
server_key: "8a6f94006ebfeab36de78c031e3f4a62"
server_key_iv: "e2f296402b8b67d066a14628d8ed5f5b"
server_key_version: "encryptionkey@0"
I20260809 01:02:33.259277 19590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:33.270574 19590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:33.274318 19590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:33.274610 19590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:33.275560 19590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:33.276858 19590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:33.276962 19590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:33.277110 19590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:33.277184 19590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:33.311165 19590 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:38429
I20260809 01:02:33.311214 19724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:38429 every 8 connection(s)
I20260809 01:02:33.312373 19590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:02:33.314863 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 19590
I20260809 01:02:33.315066 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:02:33.315717 4426 external_mini_cluster.cc:1428] Setting key a045be2a4495c09947cda62934156048
Aug 09 01:02:33 dist-test-slave-07c2 krb5kdc[19200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237353, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:33.330811 19446 heartbeater.cc:503] Master 127.4.82.190:41471 was elected leader, sending a full tablet report...
I20260809 01:02:33.333349 19448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.315069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:57279 (local address 127.4.82.190:41471)
0809 01:02:33.315331 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:33.315337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:33.316578 (+ 1241us) server_negotiation.cc:407] Connection header received
0809 01:02:33.318725 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:33.318730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:33.318813 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:33.318921 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:33.320494 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.321245 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.322707 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.323101 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.323194 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.326931 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:33.326936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:33.326939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:33.326977 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:33.329858 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:33.330814 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:33.330820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:33.330822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:33.330990 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:33.331712 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:33.331716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:33.331719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:33.331989 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:33.332151 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:33.332800 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 01:02:33.333016 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 01:02:33.334218 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.315528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:57279)
0809 01:02:33.316379 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:33.316436 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:33.318394 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:33.319171 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:33.319179 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:33.319748 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:33.320323 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:33.320341 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.321379 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.321383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:33.322578 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:33.322590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.323280 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.323336 (+ 56us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:33.323546 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.324332 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:33.324369 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:33.326755 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:33.330028 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:33.330036 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:33.330050 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:33.330611 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:33.331128 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:33.331134 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:33.331140 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:33.331566 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:33.332122 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:33.332130 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:33.332435 (+ 305us) client_negotiation.cc:769] Sending connection context
0809 01:02:33.332684 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 01:02:33.333156 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":334,"threads_started":1}
I20260809 01:02:33.335474 19725 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41471
I20260809 01:02:33.335731 19725 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:33.336338 19725 heartbeater.cc:511] Master 127.4.82.190:41471 requested a full tablet report, sending...
I20260809 01:02:33.337590 19249 ts_manager.cc:194] Registered new tserver with Master: da002cbbe93848e7a8188d112dc753c7 (127.4.82.131:38429)
I20260809 01:02:33.338450 19249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:57279
I20260809 01:02:33.343410 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:02:33.360028 19448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.348311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42078 (local address 127.4.82.190:41471)
0809 01:02:33.348456 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:33.348460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:33.348891 (+ 431us) server_negotiation.cc:407] Connection header received
0809 01:02:33.348998 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:33.349002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:33.349058 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:33.349134 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:33.350458 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.351144 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.352454 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.352866 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.352962 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.355338 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:33.355342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:33.355346 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:33.355372 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:33.357757 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:33.358348 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:33.358352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:33.358354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:33.358482 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:33.358789 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:33.358792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:33.358794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:33.359111 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:33.359372 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:33.359642 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 01:02:33.359806 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 01:02:33.363793 19249 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:02:33.370553 19249 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:02:33.396415 19738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.381780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:41017 (local address 127.0.0.1:54482)
0809 01:02:33.383635 (+ 1855us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:33.383657 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:33.383780 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:33.385191 (+ 1411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:33.385202 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:33.385244 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:33.385758 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:33.385776 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.388627 (+ 2851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.388636 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:33.391776 (+ 3140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:33.391797 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.395210 (+ 3413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.395233 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:33.395517 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.395579 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 01:02:33.395698 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 01:02:33.395966 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1676,"mutex_wait_us":27,"thread_start_us":1246,"threads_started":1}
I20260809 01:02:33.397217 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.382449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54482 (local address 127.4.82.130:41017)
0809 01:02:33.384404 (+ 1955us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:33.384416 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:33.384452 (+ 36us) server_negotiation.cc:407] Connection header received
0809 01:02:33.384578 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:33.384590 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:33.384890 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:33.386011 (+ 1121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:33.386325 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.388406 (+ 2081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.392049 (+ 3643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.394052 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.396425 (+ 2373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.396642 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:33.396705 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 01:02:33.396889 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1743,"thread_start_us":1593,"threads_started":1}
I20260809 01:02:33.400909 19518 tablet_service.cc:1467] Processing CreateTablet for tablet aaee56b21c754f1594ab3fc680ae5aba (DEFAULT_TABLE table=test-table [id=7d820787f8db4ce3bac64d2be2f590d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:02:33.402390 19518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaee56b21c754f1594ab3fc680ae5aba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:33.402486 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.381534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:35817 (local address 127.0.0.1:54524)
0809 01:02:33.382620 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:33.382665 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:33.382872 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:33.395390 (+ 12518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:33.395396 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:33.395427 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:33.395903 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:33.395916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.397957 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.397965 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:33.400144 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:33.400158 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.401623 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.401629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:33.401795 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.401824 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 01:02:33.401905 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 01:02:33.402042 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"mutex_wait_us":1,"thread_start_us":1731,"threads_started":1}
I20260809 01:02:33.403486 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.381913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54524 (local address 127.4.82.129:35817)
0809 01:02:33.394716 (+ 12803us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:33.394726 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:33.394756 (+ 30us) server_negotiation.cc:407] Connection header received
0809 01:02:33.394861 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:33.394869 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:33.395097 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:33.396122 (+ 1025us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:33.396406 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.397833 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.400310 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.401420 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.401564 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.402828 (+ 1264us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:33.402902 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 01:02:33.403111 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12623,"thread_start_us":12462,"threads_started":1}
I20260809 01:02:33.404115 19742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.383919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:38429 (local address 127.0.0.1:52274)
0809 01:02:33.395614 (+ 11695us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:33.395634 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:33.395780 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:33.396755 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:33.396762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:33.396787 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:33.397124 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:33.397135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.398994 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.398999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:33.400809 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:33.400823 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.403261 (+ 2438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.403271 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:33.403485 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.403524 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 01:02:33.403619 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 01:02:33.403746 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11537,"mutex_wait_us":18,"thread_start_us":11384,"threads_started":1}
I20260809 01:02:33.404230 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.384484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52274 (local address 127.4.82.131:38429)
0809 01:02:33.395067 (+ 10583us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:33.395074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:33.395935 (+ 861us) server_negotiation.cc:407] Connection header received
0809 01:02:33.396037 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:33.396047 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:33.396259 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:33.396442 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:33.397319 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.398832 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.400981 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.402412 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.402855 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.403089 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:33.403737 (+ 648us) server_negotiation.cc:299] Negotiation successful
0809 01:02:33.403902 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10430,"thread_start_us":10268,"threads_started":1}
I20260809 01:02:33.407258 19654 tablet_service.cc:1467] Processing CreateTablet for tablet aaee56b21c754f1594ab3fc680ae5aba (DEFAULT_TABLE table=test-table [id=7d820787f8db4ce3bac64d2be2f590d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:02:33.407191 19380 tablet_service.cc:1467] Processing CreateTablet for tablet aaee56b21c754f1594ab3fc680ae5aba (DEFAULT_TABLE table=test-table [id=7d820787f8db4ce3bac64d2be2f590d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:02:33.408435 19654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaee56b21c754f1594ab3fc680ae5aba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:33.408608 19380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aaee56b21c754f1594ab3fc680ae5aba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:33.418093 19745 tablet_bootstrap.cc:492] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Bootstrap starting.
I20260809 01:02:33.422012 19746 tablet_bootstrap.cc:492] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7: Bootstrap starting.
I20260809 01:02:33.422429 19745 tablet_bootstrap.cc:654] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:33.423252 19747 tablet_bootstrap.cc:492] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb: Bootstrap starting.
I20260809 01:02:33.424602 19745 log.cc:824] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:33.425570 19746 tablet_bootstrap.cc:654] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:33.426379 19747 tablet_bootstrap.cc:654] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:33.426748 19746 log.cc:824] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:33.427440 19747 log.cc:824] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:33.427866 19745 tablet_bootstrap.cc:492] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: No bootstrap required, opened a new log
I20260809 01:02:33.428170 19745 ts_tablet_manager.cc:1397] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 01:02:33.429876 19746 tablet_bootstrap.cc:492] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7: No bootstrap required, opened a new log
I20260809 01:02:33.430119 19746 ts_tablet_manager.cc:1397] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260809 01:02:33.431828 19747 tablet_bootstrap.cc:492] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb: No bootstrap required, opened a new log
I20260809 01:02:33.432114 19747 ts_tablet_manager.cc:1397] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 01:02:33.438972 19745 raft_consensus.cc:348] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:33.439114 19747 raft_consensus.cc:348] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:33.439528 19745 raft_consensus.cc:374] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:33.439548 19747 raft_consensus.cc:374] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:33.439635 19745 raft_consensus.cc:729] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11dd0a98f5de4e6fbc5b35ba5f6ea1f4, State: Initialized, Role: FOLLOWER
I20260809 01:02:33.439653 19747 raft_consensus.cc:729] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10a7ff25c6194e618d721e2597074ccb, State: Initialized, Role: FOLLOWER
I20260809 01:02:33.440240 19747 consensus_queue.cc:260] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:33.440371 19745 consensus_queue.cc:260] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:33.440531 19746 raft_consensus.cc:348] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:33.441359 19746 raft_consensus.cc:374] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:33.441483 19746 raft_consensus.cc:729] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da002cbbe93848e7a8188d112dc753c7, State: Initialized, Role: FOLLOWER
I20260809 01:02:33.442101 19747 ts_tablet_manager.cc:1428] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb: Time spent starting tablet: real 0.010s user 0.006s sys 0.004s
I20260809 01:02:33.442269 19746 consensus_queue.cc:260] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:33.443181 19585 heartbeater.cc:503] Master 127.4.82.190:41471 was elected leader, sending a full tablet report...
I20260809 01:02:33.444339 19745 ts_tablet_manager.cc:1428] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260809 01:02:33.445416 19725 heartbeater.cc:503] Master 127.4.82.190:41471 was elected leader, sending a full tablet report...
I20260809 01:02:33.446525 19746 ts_tablet_manager.cc:1428] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260809 01:02:33.547610 19753 raft_consensus.cc:482] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:02:33.547945 19753 raft_consensus.cc:504] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:33.549650 19753 leader_election.cc:288] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10a7ff25c6194e618d721e2597074ccb (127.4.82.129:35817), 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 (127.4.82.130:41017)
W20260809 01:02:33.549845 19447 tablet.cc:2369] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:02:33.556821 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.549876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:41017 (local address 127.4.82.131:36031)
0809 01:02:33.550188 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:33.550211 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:33.550350 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:33.551080 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:33.551086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:33.551124 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:33.551400 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:33.551411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.552725 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.552729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:33.554754 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:33.554769 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.556183 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.556187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:33.556325 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.556348 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 01:02:33.556420 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 01:02:33.556504 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260809 01:02:33.556902 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.550365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:36031 (local address 127.4.82.130:41017)
0809 01:02:33.550707 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:33.550716 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:33.550740 (+ 24us) server_negotiation.cc:407] Connection header received
0809 01:02:33.550831 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:33.550839 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:33.550923 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:33.551024 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:33.551551 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.552597 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.554917 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.556049 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.556203 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.556280 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:33.556474 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 01:02:33.556615 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260809 01:02:33.557365 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.550555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:53533 (local address 127.4.82.129:35817)
0809 01:02:33.550787 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:33.550796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:33.551271 (+ 475us) server_negotiation.cc:407] Connection header received
0809 01:02:33.551435 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:33.551439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:33.551526 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:33.551626 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:33.552085 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.552998 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.555261 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.556507 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.556655 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.556726 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:33.557078 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 01:02:33.557202 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 01:02:33.557886 19586 tablet.cc:2369] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:02:33.558077 19754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.549973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:35817 (local address 127.4.82.131:53533)
0809 01:02:33.551146 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:33.551163 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:33.551323 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:33.551640 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:33.551642 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:33.551662 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:33.551939 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:33.551945 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.553142 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.553152 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:33.555110 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:33.555119 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.556712 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.556724 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:33.556898 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.556931 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 01:02:33.557560 (+ 629us) client_negotiation.cc:240] Negotiation successful
0809 01:02:33.557683 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"mutex_wait_us":4,"thread_start_us":873,"threads_started":1}
I20260809 01:02:33.557969 19538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaee56b21c754f1594ab3fc680ae5aba" candidate_uuid: "da002cbbe93848e7a8188d112dc753c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" is_pre_election: true
I20260809 01:02:33.558630 19538 raft_consensus.cc:2454] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da002cbbe93848e7a8188d112dc753c7 in term 0.
I20260809 01:02:33.559139 19400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaee56b21c754f1594ab3fc680ae5aba" candidate_uuid: "da002cbbe93848e7a8188d112dc753c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a7ff25c6194e618d721e2597074ccb" is_pre_election: true
I20260809 01:02:33.559331 19607 leader_election.cc:302] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11dd0a98f5de4e6fbc5b35ba5f6ea1f4, da002cbbe93848e7a8188d112dc753c7; no voters:
I20260809 01:02:33.559679 19753 raft_consensus.cc:2781] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:02:33.559708 19400 raft_consensus.cc:2454] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da002cbbe93848e7a8188d112dc753c7 in term 0.
I20260809 01:02:33.559796 19753 raft_consensus.cc:482] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:02:33.559902 19753 raft_consensus.cc:3037] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:33.562222 19753 raft_consensus.cc:504] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:33.562893 19753 leader_election.cc:288] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [CANDIDATE]: Term 1 election: Requested vote from peers 10a7ff25c6194e618d721e2597074ccb (127.4.82.129:35817), 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 (127.4.82.130:41017)
I20260809 01:02:33.563271 19400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaee56b21c754f1594ab3fc680ae5aba" candidate_uuid: "da002cbbe93848e7a8188d112dc753c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a7ff25c6194e618d721e2597074ccb"
I20260809 01:02:33.563414 19538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aaee56b21c754f1594ab3fc680ae5aba" candidate_uuid: "da002cbbe93848e7a8188d112dc753c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4"
I20260809 01:02:33.563505 19400 raft_consensus.cc:3037] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:33.563603 19538 raft_consensus.cc:3037] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 0 FOLLOWER]: Advancing to term 1
W20260809 01:02:33.564265 19726 tablet.cc:2369] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:02:33.566781 19400 raft_consensus.cc:2454] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da002cbbe93848e7a8188d112dc753c7 in term 1.
I20260809 01:02:33.566792 19538 raft_consensus.cc:2454] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da002cbbe93848e7a8188d112dc753c7 in term 1.
I20260809 01:02:33.567185 19607 leader_election.cc:302] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11dd0a98f5de4e6fbc5b35ba5f6ea1f4, da002cbbe93848e7a8188d112dc753c7; no voters:
I20260809 01:02:33.567528 19753 raft_consensus.cc:2781] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:02:33.568019 19753 raft_consensus.cc:686] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [term 1 LEADER]: Becoming Leader. State: Replica: da002cbbe93848e7a8188d112dc753c7, State: Running, Role: LEADER
I20260809 01:02:33.568464 19753 consensus_queue.cc:237] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:33.573501 19248 catalog_manager.cc:5495] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to da002cbbe93848e7a8188d112dc753c7 (127.4.82.131). New cstate: current_term: 1 leader_uuid: "da002cbbe93848e7a8188d112dc753c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } health_report { overall_health: HEALTHY } } }
I20260809 01:02:33.595224 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.589977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52286 (local address 127.4.82.131:38429)
0809 01:02:33.590182 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:33.590189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:33.590209 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:02:33.590311 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:33.590314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:33.590383 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:33.590536 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:02:33.590919 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.591720 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.593130 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.593527 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.593664 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.594166 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:02:33.594628 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:02:33.594782 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:33.594878 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 01:02:33.594953 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 01:02:33.603103 19400 raft_consensus.cc:1261] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb [term 1 FOLLOWER]: Refusing update from remote peer da002cbbe93848e7a8188d112dc753c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:02:33.603103 19538 raft_consensus.cc:1261] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 1 FOLLOWER]: Refusing update from remote peer da002cbbe93848e7a8188d112dc753c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:02:33.604190 19755 consensus_queue.cc:1035] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:02:33.604545 19753 consensus_queue.cc:1035] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:02:33.616734 19758 mvcc.cc:205] Tried to move back new op lower bound from 7316428200347848704 to 7316428200220168192. Current Snapshot: MvccSnapshot[applied={T|T < 7316428200347848704}]
I20260809 01:02:33.630321 19248 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:02:33.630616 19248 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:02:33.634048 19248 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Sending DeleteTablet for 3 replicas of tablet aaee56b21c754f1594ab3fc680ae5aba
I20260809 01:02:33.635154 19380 tablet_service.cc:1514] Processing DeleteTablet for tablet aaee56b21c754f1594ab3fc680ae5aba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:02:33 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:54524
I20260809 01:02:33.635377 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 19311
I20260809 01:02:33.635545 19654 tablet_service.cc:1514] Processing DeleteTablet for tablet aaee56b21c754f1594ab3fc680ae5aba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:02:33 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:52274
I20260809 01:02:33.636245 19518 tablet_service.cc:1514] Processing DeleteTablet for tablet aaee56b21c754f1594ab3fc680ae5aba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:02:33 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:54482
I20260809 01:02:33.650861 19768 tablet_replica.cc:330] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7: stopping tablet replica
I20260809 01:02:33.651551 19772 tablet_replica.cc:330] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: stopping tablet replica
I20260809 01:02:33.652112 19767 tablet_replica.cc:330] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb: stopping tablet replica
I20260809 01:02:33.652089 19772 raft_consensus.cc:2229] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:02:33.652305 19768 raft_consensus.cc:2229] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:02:33.652494 19767 raft_consensus.cc:2229] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:02:33.652786 19772 raft_consensus.cc:2258] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:02:33.652954 19767 raft_consensus.cc:2258] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:02:33.653118 19768 raft_consensus.cc:2258] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:02:33.655061 19768 ts_tablet_manager.cc:1905] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:02:33.655612 19772 ts_tablet_manager.cc:1905] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:02:33.656049 19767 ts_tablet_manager.cc:1905] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:02:33.662628 19772 ts_tablet_manager.cc:1918] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:02:33.662802 19772 log.cc:1196] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/aaee56b21c754f1594ab3fc680ae5aba
I20260809 01:02:33.662690 19768 ts_tablet_manager.cc:1918] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:02:33.663058 19768 log.cc:1196] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/aaee56b21c754f1594ab3fc680ae5aba
I20260809 01:02:33.663488 19772 ts_tablet_manager.cc:1939] T aaee56b21c754f1594ab3fc680ae5aba P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Deleting consensus metadata
I20260809 01:02:33.663873 19768 ts_tablet_manager.cc:1939] T aaee56b21c754f1594ab3fc680ae5aba P da002cbbe93848e7a8188d112dc753c7: Deleting consensus metadata
I20260809 01:02:33.665475 19234 catalog_manager.cc:4841] TS 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 (127.4.82.130:41017): tablet aaee56b21c754f1594ab3fc680ae5aba (table test-table [id=7d820787f8db4ce3bac64d2be2f590d2]) successfully deleted
I20260809 01:02:33.665887 19234 catalog_manager.cc:4841] TS da002cbbe93848e7a8188d112dc753c7 (127.4.82.131:38429): tablet aaee56b21c754f1594ab3fc680ae5aba (table test-table [id=7d820787f8db4ce3bac64d2be2f590d2]) successfully deleted
I20260809 01:02:33.667095 19767 ts_tablet_manager.cc:1918] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:02:33.667510 19767 log.cc:1196] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/aaee56b21c754f1594ab3fc680ae5aba
I20260809 01:02:33.668208 19767 ts_tablet_manager.cc:1939] T aaee56b21c754f1594ab3fc680ae5aba P 10a7ff25c6194e618d721e2597074ccb: Deleting consensus metadata
Aug 09 01:02:33 dist-test-slave-07c2 krb5kdc[19200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 01:02:33.670356 19235 catalog_manager.cc:4841] TS 10a7ff25c6194e618d721e2597074ccb (127.4.82.129:35817): tablet aaee56b21c754f1594ab3fc680ae5aba (table test-table [id=7d820787f8db4ce3bac64d2be2f590d2]) successfully deleted
I20260809 01:02:33.676234 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.636767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54528 (local address 127.4.82.129:35817)
0809 01:02:33.636972 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:33.636980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:33.647405 (+ 10425us) server_negotiation.cc:407] Connection header received
0809 01:02:33.647726 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:33.647735 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:33.647846 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:33.647930 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:33.653326 (+ 5396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.654440 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.664446 (+ 10006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.665081 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.665196 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.670265 (+ 5069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:33.670279 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:33.670297 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:33.670351 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:33.673338 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:33.674232 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:33.674240 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:33.674242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:33.674374 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:33.674812 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:33.674816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:33.674818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:33.675169 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:33.675481 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:33.675811 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 01:02:33.675971 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 01:02:33.676937 19441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:33.766741 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 19311
I20260809 01:02:33.786811 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 19452
Aug 09 01:02:33 dist-test-slave-07c2 krb5kdc[19200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 01:02:33.804899 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.787723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54492 (local address 127.4.82.130:41017)
0809 01:02:33.787891 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:33.787895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:33.787955 (+ 60us) server_negotiation.cc:407] Connection header received
0809 01:02:33.788137 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:33.788141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:33.788207 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:33.788298 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:33.790040 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.790789 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.792798 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.793215 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.793322 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.798645 (+ 5323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:33.798658 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:33.798676 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:33.798710 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:33.801631 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:33.802452 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:33.802460 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:33.802462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:33.802581 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:33.802990 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:33.802993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:33.802995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:33.803456 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:33.803777 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:33.804061 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 01:02:33.804311 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 01:02:33.805627 19580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:33.894887 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 19452
I20260809 01:02:33.911053 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 19590
Aug 09 01:02:33 dist-test-slave-07c2 krb5kdc[19200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 01:02:33.929395 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:33.912027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52296 (local address 127.4.82.131:38429)
0809 01:02:33.912242 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:33.912254 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:33.912277 (+ 23us) server_negotiation.cc:407] Connection header received
0809 01:02:33.912344 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:33.912349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:33.912436 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:33.912544 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:33.913849 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.915143 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.916546 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:33.917210 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:33.917355 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:33.922034 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:33.922049 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:33.922069 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:33.922115 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:33.926225 (+ 4110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:33.927042 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:33.927052 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:33.927055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:33.927308 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:33.927670 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:33.927675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:33.927679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:33.928167 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:33.928483 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:33.928698 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 01:02:33.928912 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 01:02:33.930248 19720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:34.053344 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 19590
I20260809 01:02:34.075253 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 19214
I20260809 01:02:34.076454 19279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:34.187565 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 19214
I20260809 01:02:34.211617 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41471
--webserver_interface=127.4.82.190
--webserver_port=45203
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:41471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:34.550761 19777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:34.551062 19777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:34.551216 19777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:34.558459 19777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:02:34.558602 19777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:34.558707 19777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:34.558818 19777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:02:34.558899 19777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:02:34.569463 19777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:41471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=45203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:34.570849 19777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:34.572379 19777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:34.604720 19783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:34.604656 19786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:34.604616 19784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:34.605650 19777 server_base.cc:1034] running on GCE node
I20260809 01:02:34.606550 19777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:34.608318 19777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:34.609577 19777 hybrid_clock.cc:648] HybridClock initialized: now 1786237354609559 us; error 119 us; skew 500 ppm
I20260809 01:02:34.610008 19777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:34 dist-test-slave-07c2 krb5kdc[19200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237354, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:34.613595 19777 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:02:34.615375 19777 webserver.cc:460] Webserver started at http://127.4.82.190:45203/ using document root <none> and password file <none>
I20260809 01:02:34.616258 19777 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:34.616431 19777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:34.622432 19777 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 01:02:34.625375 19794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:34.626744 19777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 01:02:34.626958 19777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "fb7a947c18db4082b29f796cedd5f916"
format_stamp: "Formatted at 2026-08-09 01:02:31 on dist-test-slave-07c2"
server_key: "df59c2784ffbac4579e1f3db0cb6e9da"
server_key_iv: "501739c14d8377e0c7556eaefb30757d"
server_key_version: "encryptionkey@0"
I20260809 01:02:34.628257 19777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:34.644802 19777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:34.649011 19777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:34.649293 19777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:34.669358 19777 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:41471
I20260809 01:02:34.669376 19846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:41471 every 8 connection(s)
I20260809 01:02:34.670982 19777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:02:34.678208 19847 sys_catalog.cc:263] Verifying existing consensus state
I20260809 01:02:34.678885 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 19777
I20260809 01:02:34.679940 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:35817
--local_ip_for_outbound_sockets=127.4.82.129
--tserver_master_addrs=127.4.82.190:41471
--webserver_port=37677
--webserver_interface=127.4.82.129
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:02:34.683475 19847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Bootstrap starting.
I20260809 01:02:34.705312 19847 log.cc:824] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:34.717509 19847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 01:02:34.718036 19847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Bootstrap complete.
I20260809 01:02:34.727324 19847 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } }
I20260809 01:02:34.728191 19847 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb7a947c18db4082b29f796cedd5f916, State: Initialized, Role: FOLLOWER
I20260809 01:02:34.728863 19847 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } }
I20260809 01:02:34.729113 19847 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:02:34.729272 19847 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:02:34.729456 19847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:02:34.733765 19847 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } }
I20260809 01:02:34.734289 19847 leader_election.cc:302] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb7a947c18db4082b29f796cedd5f916; no voters:
I20260809 01:02:34.734938 19847 leader_election.cc:288] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 01:02:34.735078 19852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 01:02:34.737528 19847 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:02:34.740494 19852 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 2 LEADER]: Becoming Leader. State: Replica: fb7a947c18db4082b29f796cedd5f916, State: Running, Role: LEADER
I20260809 01:02:34.741214 19852 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } }
I20260809 01:02:34.744256 19854 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fb7a947c18db4082b29f796cedd5f916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } } }
I20260809 01:02:34.744293 19855 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb7a947c18db4082b29f796cedd5f916. Latest consensus state: current_term: 2 leader_uuid: "fb7a947c18db4082b29f796cedd5f916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } } }
I20260809 01:02:34.744637 19854 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:34.744668 19855 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:34.757139 19865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:02:34.767004 19865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:02:34.768862 19865 catalog_manager.cc:1259] Loaded cluster ID: 919aede74558473eb07f04873c05e53a
I20260809 01:02:34.768989 19865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:02:34.773159 19865 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 01:02:34.779711 19871 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 01:02:34.787968 19865 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 01:02:34.794848 19865 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:02:34.796183 19865 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Loaded TSK: 0
I20260809 01:02:34.797644 19865 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 01:02:35.054666 19849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:35.054979 19849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:35.055161 19849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:35.062376 19849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:35.062515 19849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:35.062685 19849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:02:35.072295 19849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:35817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=37677
--tserver_master_addrs=127.4.82.190:41471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:35.073555 19849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:35.074878 19849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:35.088619 19878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:35.088665 19879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:35.088914 19881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:35.090226 19849 server_base.cc:1034] running on GCE node
I20260809 01:02:35.090961 19849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:35.092207 19849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:35.093456 19849 hybrid_clock.cc:648] HybridClock initialized: now 1786237355093399 us; error 44 us; skew 500 ppm
I20260809 01:02:35.093856 19849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:35 dist-test-slave-07c2 krb5kdc[19200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237355, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:35.098001 19849 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:02:35.099581 19849 webserver.cc:460] Webserver started at http://127.4.82.129:37677/ using document root <none> and password file <none>
I20260809 01:02:35.100284 19849 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:35.100441 19849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:35.105825 19849 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 01:02:35.108693 19888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:35.110034 19849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 01:02:35.110214 19849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "10a7ff25c6194e618d721e2597074ccb"
format_stamp: "Formatted at 2026-08-09 01:02:32 on dist-test-slave-07c2"
server_key: "bc0e5bad69ee34e73976ac362440aa69"
server_key_iv: "b617926b9e39cc1ee59ceb81ba4ff596"
server_key_version: "encryptionkey@0"
I20260809 01:02:35.111158 19849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:35.123720 19849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:35.127837 19849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:35.128141 19849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:35.129115 19849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:35.130494 19849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:35.130625 19849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:35.130746 19849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:35.130831 19849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:35.165840 19849 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:35817
I20260809 01:02:35.165925 20006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:35817 every 8 connection(s)
I20260809 01:02:35.167336 19849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:02:35.169282 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 19849
I20260809 01:02:35.169924 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:41017
--local_ip_for_outbound_sockets=127.4.82.130
--tserver_master_addrs=127.4.82.190:41471
--webserver_port=46269
--webserver_interface=127.4.82.130
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 01:02:35 dist-test-slave-07c2 krb5kdc[19200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237355, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:35.193946 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:35.170503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:34027)
0809 01:02:35.171551 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:35.171629 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:35.174272 (+ 2643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:35.175607 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:35.175620 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:35.176380 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:35.177140 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:35.177165 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:35.178803 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:35.178811 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:35.180559 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:35.180575 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:35.182015 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:35.182020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:35.182191 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:35.182859 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:35.182880 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:35.185235 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:35.189914 (+ 4679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:35.189920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:35.189929 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:35.190352 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:35.190836 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:35.190839 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:35.190841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:35.191001 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:35.191987 (+ 986us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:35.191992 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:35.192281 (+ 289us) client_negotiation.cc:769] Sending connection context
0809 01:02:35.192506 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 01:02:35.192951 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":492,"threads_started":1}
I20260809 01:02:35.194540 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:35.170381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:34027 (local address 127.4.82.190:41471)
0809 01:02:35.171064 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:35.171084 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:35.171767 (+ 683us) server_negotiation.cc:407] Connection header received
0809 01:02:35.174781 (+ 3014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:35.174793 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:35.175087 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:35.175439 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:35.177447 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:35.178647 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:35.180713 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:35.181795 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:35.181991 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:35.185494 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:35.185507 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:35.185526 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:35.185580 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:35.189769 (+ 4189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:35.190497 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:35.190508 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:35.190518 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:35.190724 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:35.191124 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:35.191132 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:35.191138 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:35.191823 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:35.192042 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:35.192777 (+ 735us) server_negotiation.cc:299] Negotiation successful
0809 01:02:35.193235 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"spinlock_wait_cycles":5504,"thread_start_us":371,"threads_started":1}
I20260809 01:02:35.197116 20007 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41471
I20260809 01:02:35.197476 20007 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:35.198467 20007 heartbeater.cc:511] Master 127.4.82.190:41471 requested a full tablet report, sending...
I20260809 01:02:35.200750 19812 ts_manager.cc:194] Registered new tserver with Master: 10a7ff25c6194e618d721e2597074ccb (127.4.82.129:35817)
I20260809 01:02:35.202261 19812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:34027
W20260809 01:02:35.505712 20011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:35.506011 20011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:35.506181 20011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:35.513378 20011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:35.513527 20011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:35.513705 20011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:02:35.523393 20011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:41017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=46269
--tserver_master_addrs=127.4.82.190:41471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:35.524609 20011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:35.525892 20011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:35.539397 20016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:35.539434 20017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:35.539599 20019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:35.540495 20011 server_base.cc:1034] running on GCE node
I20260809 01:02:35.541111 20011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:35.542146 20011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:35.543385 20011 hybrid_clock.cc:648] HybridClock initialized: now 1786237355543334 us; error 46 us; skew 500 ppm
I20260809 01:02:35.543701 20011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:35 dist-test-slave-07c2 krb5kdc[19200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237355, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:35.546824 20011 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:02:35.548290 20011 webserver.cc:460] Webserver started at http://127.4.82.130:46269/ using document root <none> and password file <none>
I20260809 01:02:35.548952 20011 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:35.549067 20011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:35.554155 20011 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260809 01:02:35.557185 20026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:35.558439 20011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 01:02:35.558614 20011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4"
format_stamp: "Formatted at 2026-08-09 01:02:32 on dist-test-slave-07c2"
server_key: "effc839db13e3fdde54dcf47d29f412d"
server_key_iv: "d75f7ebf24f547d2cb2e2695b838a0fa"
server_key_version: "encryptionkey@0"
I20260809 01:02:35.559644 20011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:35.576411 20011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:35.581025 20011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:35.581317 20011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:35.582278 20011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:35.583689 20011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:35.583774 20011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:35.583918 20011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:35.584012 20011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:35.617707 20011 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:41017
I20260809 01:02:35.617810 20146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:41017 every 8 connection(s)
I20260809 01:02:35.619105 20011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:02:35.620085 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 20011
I20260809 01:02:35.620798 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:38429
--local_ip_for_outbound_sockets=127.4.82.131
--tserver_master_addrs=127.4.82.190:41471
--webserver_port=34771
--webserver_interface=127.4.82.131
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 01:02:35 dist-test-slave-07c2 krb5kdc[19200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237355, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:35.640043 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:35.621751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:57335 (local address 127.4.82.190:41471)
0809 01:02:35.621928 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:35.621932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:35.623350 (+ 1418us) server_negotiation.cc:407] Connection header received
0809 01:02:35.625198 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:35.625204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:35.625298 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:35.625412 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:35.626810 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:35.627557 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:35.629117 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:35.629530 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:35.629676 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:35.634092 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:35.634097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:35.634100 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:35.634137 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:35.636752 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:35.637710 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:35.637714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:35.637716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:35.637841 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:35.638340 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:35.638343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:35.638345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:35.638592 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:35.638775 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:35.639643 (+ 868us) server_negotiation.cc:299] Negotiation successful
0809 01:02:35.639804 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 01:02:35.641453 20149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:35.622376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:57335)
0809 01:02:35.623154 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:35.623197 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:35.624922 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:35.625653 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:35.625661 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:35.626113 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:35.626629 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:35.626644 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:35.627713 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:35.627725 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:35.628940 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:35.628952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:35.629775 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:35.629782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:35.630006 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:35.630758 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:35.630793 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:35.633886 (+ 3093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:35.636894 (+ 3008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:35.636906 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:35.636924 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:35.637538 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:35.637959 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:35.637967 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:35.637973 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:35.638230 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:35.638747 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:35.638757 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:35.639209 (+ 452us) client_negotiation.cc:769] Sending connection context
0809 01:02:35.639542 (+ 333us) client_negotiation.cc:240] Negotiation successful
0809 01:02:35.640044 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":323,"threads_started":1}
I20260809 01:02:35.642951 20147 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41471
I20260809 01:02:35.643275 20147 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:35.644148 20147 heartbeater.cc:511] Master 127.4.82.190:41471 requested a full tablet report, sending...
I20260809 01:02:35.645824 19812 ts_manager.cc:194] Registered new tserver with Master: 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 (127.4.82.130:41017)
I20260809 01:02:35.647141 19812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:57335
W20260809 01:02:35.952112 20150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:35.952360 20150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:35.952450 20150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:35.959399 20150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:35.959489 20150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:35.959591 20150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:02:35.968776 20150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:38429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=34771
--tserver_master_addrs=127.4.82.190:41471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:35.969841 20150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:35.971004 20150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:35.983668 20155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:35.983729 20158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:35.983768 20156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:35.984447 20150 server_base.cc:1034] running on GCE node
I20260809 01:02:35.984946 20150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:35.985786 20150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:35.987004 20150 hybrid_clock.cc:648] HybridClock initialized: now 1786237355986968 us; error 47 us; skew 500 ppm
I20260809 01:02:35.987349 20150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:35 dist-test-slave-07c2 krb5kdc[19200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237355, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:35.990299 20150 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:02:35.991772 20150 webserver.cc:460] Webserver started at http://127.4.82.131:34771/ using document root <none> and password file <none>
I20260809 01:02:35.992424 20150 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:35.992592 20150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:35.997440 20150 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 01:02:35.999995 20166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:36.001219 20150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:02:36.001349 20150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "da002cbbe93848e7a8188d112dc753c7"
format_stamp: "Formatted at 2026-08-09 01:02:33 on dist-test-slave-07c2"
server_key: "8a6f94006ebfeab36de78c031e3f4a62"
server_key_iv: "e2f296402b8b67d066a14628d8ed5f5b"
server_key_version: "encryptionkey@0"
I20260809 01:02:36.002228 20150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:36.018520 20150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:36.022239 20150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:36.022451 20150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:36.023388 20150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:36.024609 20150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:36.024703 20150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:36.024776 20150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:36.024834 20150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:36.059583 20150 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:38429
I20260809 01:02:36.059665 20283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:38429 every 8 connection(s)
I20260809 01:02:36.061188 20150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:02:36.069406 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 20150
Aug 09 01:02:36 dist-test-slave-07c2 krb5kdc[19200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237355, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:36.082151 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.063740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:45065 (local address 127.4.82.190:41471)
0809 01:02:36.063921 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:36.063926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:36.065036 (+ 1110us) server_negotiation.cc:407] Connection header received
0809 01:02:36.066747 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:36.066750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:36.066817 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:36.066903 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:36.068339 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.069084 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.071184 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.071955 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.072099 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.075497 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:36.075503 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:36.075509 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:36.075554 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:36.078972 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:36.079810 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:36.079813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:36.079816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:36.079953 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:36.080457 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:36.080460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:36.080462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:36.080765 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:36.080879 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:36.081759 (+ 880us) server_negotiation.cc:299] Negotiation successful
0809 01:02:36.081913 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 01:02:36.083385 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.064193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:45065)
0809 01:02:36.064868 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:36.064910 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:36.066476 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:36.067096 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:36.067103 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:36.067673 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:36.068188 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:36.068204 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.069255 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.069260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:36.071036 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:36.071048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.072201 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.072210 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:36.072434 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.073085 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:36.073105 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:36.075217 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:36.079100 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:36.079106 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:36.079116 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:36.079687 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:36.080070 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:36.080077 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:36.080083 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:36.080316 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:36.080900 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:36.080909 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:36.081330 (+ 421us) client_negotiation.cc:769] Sending connection context
0809 01:02:36.081629 (+ 299us) client_negotiation.cc:240] Negotiation successful
0809 01:02:36.082133 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":269,"threads_started":1}
I20260809 01:02:36.084884 20284 heartbeater.cc:347] Connected to a master server at 127.4.82.190:41471
I20260809 01:02:36.085191 20284 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:36.085592 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.070357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42080 (local address 127.4.82.190:41471)
0809 01:02:36.070863 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:36.070870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:36.071383 (+ 513us) server_negotiation.cc:407] Connection header received
0809 01:02:36.071449 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:36.071455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:36.071537 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:36.071663 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:36.073363 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.074101 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.075599 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.076042 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.076175 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.079487 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:36.079493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:36.079498 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:36.079534 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:36.082717 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:36.083378 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:36.083384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:36.083387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:36.083527 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:36.084108 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:36.084116 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:36.084120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:36.084449 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:36.084685 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:36.085007 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 01:02:36.085183 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":264,"threads_started":1}
I20260809 01:02:36.086026 20284 heartbeater.cc:511] Master 127.4.82.190:41471 requested a full tablet report, sending...
I20260809 01:02:36.087481 19811 ts_manager.cc:194] Registered new tserver with Master: da002cbbe93848e7a8188d112dc753c7 (127.4.82.131:38429)
I20260809 01:02:36.088747 19811 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:45065
I20260809 01:02:36.100298 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:02:36.106478 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.101146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42086 (local address 127.4.82.190:41471)
0809 01:02:36.101340 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:36.101349 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:36.101701 (+ 352us) server_negotiation.cc:407] Connection header received
0809 01:02:36.101847 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:36.101851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:36.101960 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:36.102073 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:02:36.102437 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.103056 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.104525 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.104922 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.105009 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.105516 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:02:36.105952 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:02:36.106047 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:36.106146 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 01:02:36.106205 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 01:02:36.107424 19811 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:42086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:02:36.114450 19811 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:02:36.136412 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.125406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54500 (local address 127.4.82.130:41017)
0809 01:02:36.126261 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:36.126272 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:36.126778 (+ 506us) server_negotiation.cc:407] Connection header received
0809 01:02:36.127177 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:36.127188 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:36.127464 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:36.127613 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:36.129518 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.131089 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.133740 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.135233 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.135389 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.135605 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:36.135884 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 01:02:36.136047 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":507,"threads_started":1}
I20260809 01:02:36.136478 20297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.125261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:41017 (local address 127.0.0.1:54500)
0809 01:02:36.126649 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:36.126688 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:36.126924 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:36.128001 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:36.128010 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:36.128916 (+ 906us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:36.129339 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:36.129354 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.131237 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.131242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:36.133445 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:36.133457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.135458 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.135465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:36.135652 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.135700 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 01:02:36.135843 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 01:02:36.136011 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1181,"spinlock_wait_cycles":1903488,"thread_start_us":1044,"threads_started":1}
I20260809 01:02:36.137198 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.125302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:35817 (local address 127.0.0.1:54530)
0809 01:02:36.127133 (+ 1831us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:36.127153 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:36.128867 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:36.129787 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:36.129791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:36.129815 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:36.130245 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:36.130257 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.131917 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.131925 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:36.134236 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:36.134255 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.136288 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.136294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:36.136489 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.136529 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 01:02:36.136621 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 01:02:36.136768 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1710,"mutex_wait_us":2,"thread_start_us":1582,"threads_started":1}
I20260809 01:02:36.137727 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.127029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:38429 (local address 127.0.0.1:52306)
0809 01:02:36.129160 (+ 2131us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:36.129177 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:36.129299 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:36.129806 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:36.129810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:36.129836 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:36.130580 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:36.130591 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.132600 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.132605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:36.134869 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:36.134886 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.136920 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.136928 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:36.137080 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.137102 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 01:02:36.137228 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 01:02:36.137366 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1985,"mutex_wait_us":4,"thread_start_us":1831,"threads_started":1}
I20260809 01:02:36.138063 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.127566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52306 (local address 127.4.82.131:38429)
0809 01:02:36.128713 (+ 1147us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:36.128721 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:36.129285 (+ 564us) server_negotiation.cc:407] Connection header received
0809 01:02:36.129464 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:36.129471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:36.129629 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:36.129818 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:36.130801 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.132443 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.135011 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.136387 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.136532 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.136749 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:36.137442 (+ 693us) server_negotiation.cc:299] Negotiation successful
0809 01:02:36.137598 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":990,"thread_start_us":836,"threads_started":1}
I20260809 01:02:36.140122 20078 tablet_service.cc:1467] Processing CreateTablet for tablet 1e5f1a11fbe1465d951ed99b60e797c2 (DEFAULT_TABLE table=test-table [id=2c11e88cb85f4d4f85255d775a7ff746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:02:36.140929 20215 tablet_service.cc:1467] Processing CreateTablet for tablet 1e5f1a11fbe1465d951ed99b60e797c2 (DEFAULT_TABLE table=test-table [id=2c11e88cb85f4d4f85255d775a7ff746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:02:36.141323 20078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e5f1a11fbe1465d951ed99b60e797c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:36.142148 20215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e5f1a11fbe1465d951ed99b60e797c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:36.147672 20299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.125830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54530 (local address 127.4.82.129:35817)
0809 01:02:36.126671 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:36.126682 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:36.127252 (+ 570us) server_negotiation.cc:407] Connection header received
0809 01:02:36.129106 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:36.129113 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:36.129342 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:36.129499 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:36.130468 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.131780 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.134429 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.135816 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.135919 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.136078 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:36.137451 (+ 1373us) server_negotiation.cc:299] Negotiation successful
0809 01:02:36.137626 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":563,"threads_started":1}
I20260809 01:02:36.151151 19938 tablet_service.cc:1467] Processing CreateTablet for tablet 1e5f1a11fbe1465d951ed99b60e797c2 (DEFAULT_TABLE table=test-table [id=2c11e88cb85f4d4f85255d775a7ff746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:02:36.152405 19938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e5f1a11fbe1465d951ed99b60e797c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:36.153946 20302 tablet_bootstrap.cc:492] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Bootstrap starting.
I20260809 01:02:36.155838 20303 tablet_bootstrap.cc:492] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7: Bootstrap starting.
I20260809 01:02:36.158406 20302 tablet_bootstrap.cc:654] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:36.159197 20303 tablet_bootstrap.cc:654] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:36.159710 20302 log.cc:824] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:36.160444 20303 log.cc:824] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:36.162961 20302 tablet_bootstrap.cc:492] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: No bootstrap required, opened a new log
I20260809 01:02:36.163225 20302 ts_tablet_manager.cc:1397] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260809 01:02:36.165096 20303 tablet_bootstrap.cc:492] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7: No bootstrap required, opened a new log
I20260809 01:02:36.165407 20303 ts_tablet_manager.cc:1397] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260809 01:02:36.173888 20302 raft_consensus.cc:348] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:36.174342 20302 raft_consensus.cc:374] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:36.174451 20302 raft_consensus.cc:729] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11dd0a98f5de4e6fbc5b35ba5f6ea1f4, State: Initialized, Role: FOLLOWER
I20260809 01:02:36.175146 20302 consensus_queue.cc:260] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:36.176247 20303 raft_consensus.cc:348] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:36.176724 20303 raft_consensus.cc:374] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:36.176875 20303 raft_consensus.cc:729] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da002cbbe93848e7a8188d112dc753c7, State: Initialized, Role: FOLLOWER
I20260809 01:02:36.176987 20307 tablet_bootstrap.cc:492] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb: Bootstrap starting.
I20260809 01:02:36.178411 20303 consensus_queue.cc:260] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:36.180964 20302 ts_tablet_manager.cc:1428] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Time spent starting tablet: real 0.018s user 0.012s sys 0.003s
I20260809 01:02:36.181313 20147 heartbeater.cc:503] Master 127.4.82.190:41471 was elected leader, sending a full tablet report...
I20260809 01:02:36.183187 20307 tablet_bootstrap.cc:654] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:36.183320 20284 heartbeater.cc:503] Master 127.4.82.190:41471 was elected leader, sending a full tablet report...
I20260809 01:02:36.184105 20303 ts_tablet_manager.cc:1428] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7: Time spent starting tablet: real 0.019s user 0.008s sys 0.008s
I20260809 01:02:36.184760 20307 log.cc:824] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:36.188284 20307 tablet_bootstrap.cc:492] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb: No bootstrap required, opened a new log
I20260809 01:02:36.188530 20307 ts_tablet_manager.cc:1397] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260809 01:02:36.194888 20307 raft_consensus.cc:348] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:36.195271 20307 raft_consensus.cc:374] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:36.195415 20307 raft_consensus.cc:729] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10a7ff25c6194e618d721e2597074ccb, State: Initialized, Role: FOLLOWER
I20260809 01:02:36.195978 20307 consensus_queue.cc:260] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:36.196960 20007 heartbeater.cc:503] Master 127.4.82.190:41471 was elected leader, sending a full tablet report...
I20260809 01:02:36.197501 20307 ts_tablet_manager.cc:1428] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb: Time spent starting tablet: real 0.009s user 0.006s sys 0.003s
I20260809 01:02:36.262084 20309 raft_consensus.cc:482] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:02:36.262405 20309 raft_consensus.cc:504] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:36.264261 20309 leader_election.cc:288] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 (127.4.82.130:41017), 10a7ff25c6194e618d721e2597074ccb (127.4.82.129:35817)
I20260809 01:02:36.269636 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.264404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:41017 (local address 127.4.82.131:54493)
0809 01:02:36.264593 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:36.264615 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:36.264731 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:36.265249 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:36.265260 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:36.265284 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:36.265434 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:36.265441 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.266439 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.266451 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:36.267771 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:36.267785 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.268892 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.268898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:36.269099 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.269123 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 01:02:36.269236 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 01:02:36.269367 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 01:02:36.269631 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.264699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:54493 (local address 127.4.82.130:41017)
0809 01:02:36.264913 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:36.264922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:36.264942 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:02:36.265020 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:36.265028 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:36.265104 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:36.265202 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:36.265586 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.266286 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.267938 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.268725 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.268903 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.268979 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:36.269263 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 01:02:36.269386 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 01:02:36.270443 20299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.264708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:52895 (local address 127.4.82.129:35817)
0809 01:02:36.264967 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:36.264976 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:36.265465 (+ 489us) server_negotiation.cc:407] Connection header received
0809 01:02:36.265645 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:36.265649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:36.265712 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:36.265815 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:36.266330 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.267003 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.268778 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.269532 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.269717 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.269788 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:36.270112 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 01:02:36.270247 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 01:02:36.270511 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.264610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:35817 (local address 127.4.82.131:52895)
0809 01:02:36.265339 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:36.265356 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:36.265508 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:36.265832 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:36.265836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:36.265858 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:36.266172 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:36.266180 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.267154 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.267166 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:36.268623 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:36.268635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.269749 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.269755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:36.269938 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.269965 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 01:02:36.270051 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 01:02:36.270170 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":361,"threads_started":1}
I20260809 01:02:36.270834 20099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e5f1a11fbe1465d951ed99b60e797c2" candidate_uuid: "da002cbbe93848e7a8188d112dc753c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" is_pre_election: true
I20260809 01:02:36.271394 20099 raft_consensus.cc:2454] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da002cbbe93848e7a8188d112dc753c7 in term 0.
I20260809 01:02:36.271533 19960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e5f1a11fbe1465d951ed99b60e797c2" candidate_uuid: "da002cbbe93848e7a8188d112dc753c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a7ff25c6194e618d721e2597074ccb" is_pre_election: true
I20260809 01:02:36.272059 19960 raft_consensus.cc:2454] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da002cbbe93848e7a8188d112dc753c7 in term 0.
I20260809 01:02:36.272073 20168 leader_election.cc:302] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11dd0a98f5de4e6fbc5b35ba5f6ea1f4, da002cbbe93848e7a8188d112dc753c7; no voters:
I20260809 01:02:36.272439 20309 raft_consensus.cc:2781] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:02:36.272605 20309 raft_consensus.cc:482] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:02:36.272732 20309 raft_consensus.cc:3037] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:36.274912 20309 raft_consensus.cc:504] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:36.275518 20309 leader_election.cc:288] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [CANDIDATE]: Term 1 election: Requested vote from peers 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 (127.4.82.130:41017), 10a7ff25c6194e618d721e2597074ccb (127.4.82.129:35817)
I20260809 01:02:36.275817 20099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e5f1a11fbe1465d951ed99b60e797c2" candidate_uuid: "da002cbbe93848e7a8188d112dc753c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4"
I20260809 01:02:36.275955 19960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e5f1a11fbe1465d951ed99b60e797c2" candidate_uuid: "da002cbbe93848e7a8188d112dc753c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10a7ff25c6194e618d721e2597074ccb"
I20260809 01:02:36.276028 20099 raft_consensus.cc:3037] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:36.276180 19960 raft_consensus.cc:3037] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:36.278209 20099 raft_consensus.cc:2454] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da002cbbe93848e7a8188d112dc753c7 in term 1.
I20260809 01:02:36.278290 19960 raft_consensus.cc:2454] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da002cbbe93848e7a8188d112dc753c7 in term 1.
I20260809 01:02:36.278630 20168 leader_election.cc:302] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11dd0a98f5de4e6fbc5b35ba5f6ea1f4, da002cbbe93848e7a8188d112dc753c7; no voters:
I20260809 01:02:36.278964 20309 raft_consensus.cc:2781] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:02:36.279390 20309 raft_consensus.cc:686] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [term 1 LEADER]: Becoming Leader. State: Replica: da002cbbe93848e7a8188d112dc753c7, State: Running, Role: LEADER
I20260809 01:02:36.279821 20309 consensus_queue.cc:237] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } } peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } }
I20260809 01:02:36.284781 19812 catalog_manager.cc:5495] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to da002cbbe93848e7a8188d112dc753c7 (127.4.82.131). New cstate: current_term: 1 leader_uuid: "da002cbbe93848e7a8188d112dc753c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da002cbbe93848e7a8188d112dc753c7" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 38429 } health_report { overall_health: HEALTHY } } }
W20260809 01:02:36.312942 20285 tablet.cc:2369] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:02:36.332139 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.327084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52318 (local address 127.4.82.131:38429)
0809 01:02:36.327271 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:36.327275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:36.327338 (+ 63us) server_negotiation.cc:407] Connection header received
0809 01:02:36.327558 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:36.327561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:36.327632 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:36.327748 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:02:36.328058 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.328800 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.330149 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.330530 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.330617 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.331065 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:02:36.331547 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:02:36.331648 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:36.331755 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 01:02:36.331862 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 01:02:36.339931 20099 raft_consensus.cc:1261] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 1 FOLLOWER]: Refusing update from remote peer da002cbbe93848e7a8188d112dc753c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:02:36.339928 19960 raft_consensus.cc:1261] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb [term 1 FOLLOWER]: Refusing update from remote peer da002cbbe93848e7a8188d112dc753c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:02:36.340852 20313 consensus_queue.cc:1035] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:02:36.341214 20309 consensus_queue.cc:1035] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10a7ff25c6194e618d721e2597074ccb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 35817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:02:36.352470 20315 mvcc.cc:205] Tried to move back new op lower bound from 7316428211558760448 to 7316428211325865984. Current Snapshot: MvccSnapshot[applied={T|T < 7316428211558760448}]
I20260809 01:02:36.366542 19812 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:42086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:02:36.366851 19812 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:42086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:02:36.370594 19812 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Sending DeleteTablet for 3 replicas of tablet 1e5f1a11fbe1465d951ed99b60e797c2
I20260809 01:02:36.371771 20078 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e5f1a11fbe1465d951ed99b60e797c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:02:36 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:54500
I20260809 01:02:36.371903 19937 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e5f1a11fbe1465d951ed99b60e797c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:02:36 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:54530
I20260809 01:02:36.372116 20215 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e5f1a11fbe1465d951ed99b60e797c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:02:36 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:52306
I20260809 01:02:36.372430 19938 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e5f1a11fbe1465d951ed99b60e797c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:02:36 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:54530
I20260809 01:02:36.373422 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 19849
I20260809 01:02:36.387666 20325 tablet_replica.cc:330] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: stopping tablet replica
I20260809 01:02:36.387146 20326 tablet_replica.cc:330] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb: stopping tablet replica
I20260809 01:02:36.388079 20325 raft_consensus.cc:2229] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:02:36.388273 20326 raft_consensus.cc:2229] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:02:36.388581 20325 raft_consensus.cc:2258] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:02:36.388731 20326 raft_consensus.cc:2258] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:02:36.390919 20326 ts_tablet_manager.cc:1905] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:02:36.392592 20325 ts_tablet_manager.cc:1905] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:02:36.397863 20326 ts_tablet_manager.cc:1918] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:02:36.398017 20326 log.cc:1196] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/1e5f1a11fbe1465d951ed99b60e797c2
I20260809 01:02:36.398051 20325 ts_tablet_manager.cc:1918] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:02:36.398232 20325 log.cc:1196] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/1e5f1a11fbe1465d951ed99b60e797c2
I20260809 01:02:36.398530 20326 ts_tablet_manager.cc:1939] T 1e5f1a11fbe1465d951ed99b60e797c2 P 10a7ff25c6194e618d721e2597074ccb: Deleting consensus metadata
I20260809 01:02:36.398839 20325 ts_tablet_manager.cc:1939] T 1e5f1a11fbe1465d951ed99b60e797c2 P 11dd0a98f5de4e6fbc5b35ba5f6ea1f4: Deleting consensus metadata
I20260809 01:02:36.399788 20299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.374475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54542 (local address 127.4.82.129:35817)
0809 01:02:36.374667 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:36.374677 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:36.374878 (+ 201us) server_negotiation.cc:407] Connection header received
0809 01:02:36.375203 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:36.375212 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:36.375345 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:36.375495 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:36.377658 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.379254 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.381660 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.382551 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.382708 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.387026 (+ 4318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:36.387038 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:36.387053 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:36.387086 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:36.391696 (+ 4610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:36.396284 (+ 4588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:36.396294 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:36.396298 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:36.396443 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:36.396946 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:36.396952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:36.396955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:36.397461 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:36.397765 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:36.399134 (+ 1369us) server_negotiation.cc:299] Negotiation successful
0809 01:02:36.399420 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 01:02:36.400307 19797 catalog_manager.cc:4841] TS 10a7ff25c6194e618d721e2597074ccb (127.4.82.129:35817): tablet 1e5f1a11fbe1465d951ed99b60e797c2 (table test-table [id=2c11e88cb85f4d4f85255d775a7ff746]) successfully deleted
I20260809 01:02:36.400471 19796 catalog_manager.cc:4841] TS 11dd0a98f5de4e6fbc5b35ba5f6ea1f4 (127.4.82.130:41017): tablet 1e5f1a11fbe1465d951ed99b60e797c2 (table test-table [id=2c11e88cb85f4d4f85255d775a7ff746]) successfully deleted
W20260809 01:02:36.401315 19797 catalog_manager.cc:4808] TS 10a7ff25c6194e618d721e2597074ccb (127.4.82.129:35817): delete failed for tablet 1e5f1a11fbe1465d951ed99b60e797c2 because the tablet was not found. No further retry: Not found: Tablet not found: 1e5f1a11fbe1465d951ed99b60e797c2
I20260809 01:02:36.401772 20002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:36.404577 20327 tablet_replica.cc:330] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7: stopping tablet replica
I20260809 01:02:36.405010 20327 raft_consensus.cc:2229] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:02:36.405616 20327 raft_consensus.cc:2258] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:02:36.407250 20327 ts_tablet_manager.cc:1905] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:02:36.412216 20327 ts_tablet_manager.cc:1918] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:02:36.412397 20327 log.cc:1196] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/1e5f1a11fbe1465d951ed99b60e797c2
I20260809 01:02:36.412911 20327 ts_tablet_manager.cc:1939] T 1e5f1a11fbe1465d951ed99b60e797c2 P da002cbbe93848e7a8188d112dc753c7: Deleting consensus metadata
I20260809 01:02:36.414186 19796 catalog_manager.cc:4841] TS da002cbbe93848e7a8188d112dc753c7 (127.4.82.131:38429): tablet 1e5f1a11fbe1465d951ed99b60e797c2 (table test-table [id=2c11e88cb85f4d4f85255d775a7ff746]) successfully deleted
I20260809 01:02:36.516517 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 19849
I20260809 01:02:36.538018 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 20011
I20260809 01:02:36.554986 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.539106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54506 (local address 127.4.82.130:41017)
0809 01:02:36.539340 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:36.539346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:36.539397 (+ 51us) server_negotiation.cc:407] Connection header received
0809 01:02:36.539452 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:36.539455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:36.539534 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:36.539611 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:36.541474 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.542329 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.544508 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.544901 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.544997 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.548441 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:36.548453 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:36.548468 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:36.548500 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:36.551954 (+ 3454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:36.552905 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:36.552912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:36.552914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:36.553036 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:36.553460 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:36.553464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:36.553466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:36.553819 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:36.554035 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:36.554402 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 01:02:36.554565 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 01:02:36.555764 20141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:36.645287 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 20011
I20260809 01:02:36.664659 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 20150
I20260809 01:02:36.685359 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:36.665595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.4.82.131:38429)
0809 01:02:36.665813 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:36.665819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:36.665840 (+ 21us) server_negotiation.cc:407] Connection header received
0809 01:02:36.665908 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:36.665913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:36.666004 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:36.666126 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:36.667567 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.668445 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.669979 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:36.670431 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:36.670576 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:36.679080 (+ 8504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:36.679092 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:36.679106 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:36.679146 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:36.682159 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:36.682848 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:36.682860 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:36.682862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:36.683032 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:36.683483 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:36.683488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:36.683493 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:36.683861 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:36.684195 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:36.684826 (+ 631us) server_negotiation.cc:299] Negotiation successful
0809 01:02:36.685052 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 01:02:36.686105 20278 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:36.804111 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 20150
I20260809 01:02:36.826242 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 19777
I20260809 01:02:36.827446 19842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:36.938512 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 19777
I20260809 01:02:36.962889 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41471
--webserver_interface=127.4.82.190
--webserver_port=45203
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:41471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:37.290836 20332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:37.291172 20332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:37.291355 20332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:37.298338 20332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:02:37.298475 20332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:37.298583 20332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:37.298691 20332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:02:37.298763 20332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:02:37.307945 20332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:41471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=45203
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:37.309217 20332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:37.310509 20332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:37.322165 20337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:37.322140 20338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:37.322098 20340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:37.322814 20332 server_base.cc:1034] running on GCE node
I20260809 01:02:37.323587 20332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:37.324985 20332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:37.326231 20332 hybrid_clock.cc:648] HybridClock initialized: now 1786237357326164 us; error 55 us; skew 500 ppm
I20260809 01:02:37.326570 20332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:37 dist-test-slave-07c2 krb5kdc[19200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237357, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:37.329645 20332 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:02:37.330965 20332 webserver.cc:460] Webserver started at http://127.4.82.190:45203/ using document root <none> and password file <none>
I20260809 01:02:37.331627 20332 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:37.331737 20332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:37.337239 20332 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 01:02:37.339911 20347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:37.341163 20332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 01:02:37.341329 20332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "fb7a947c18db4082b29f796cedd5f916"
format_stamp: "Formatted at 2026-08-09 01:02:31 on dist-test-slave-07c2"
server_key: "df59c2784ffbac4579e1f3db0cb6e9da"
server_key_iv: "501739c14d8377e0c7556eaefb30757d"
server_key_version: "encryptionkey@0"
I20260809 01:02:37.342267 20332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:37.354977 20332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:37.359033 20332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:37.359359 20332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:37.377334 20332 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:41471
I20260809 01:02:37.377353 20401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:41471 every 8 connection(s)
I20260809 01:02:37.378775 20332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:02:37.385239 20402 sys_catalog.cc:263] Verifying existing consensus state
I20260809 01:02:37.386669 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 20332
I20260809 01:02:37.387533 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:35817
--local_ip_for_outbound_sockets=127.4.82.129
--tserver_master_addrs=127.4.82.190:41471
--webserver_port=37677
--webserver_interface=127.4.82.129
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:02:37.388496 20402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Bootstrap starting.
I20260809 01:02:37.411773 20402 log.cc:824] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:37.427882 20402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 01:02:37.428429 20402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916: Bootstrap complete.
I20260809 01:02:37.437304 20402 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } }
I20260809 01:02:37.438149 20402 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb7a947c18db4082b29f796cedd5f916, State: Initialized, Role: FOLLOWER
I20260809 01:02:37.438782 20402 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } }
I20260809 01:02:37.439025 20402 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:02:37.439152 20402 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:02:37.439378 20402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 2 FOLLOWER]: Advancing to term 3
I20260809 01:02:37.442920 20402 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } }
I20260809 01:02:37.443447 20402 leader_election.cc:302] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb7a947c18db4082b29f796cedd5f916; no voters:
I20260809 01:02:37.444115 20402 leader_election.cc:288] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 01:02:37.444247 20406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 01:02:37.445538 20406 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [term 3 LEADER]: Becoming Leader. State: Replica: fb7a947c18db4082b29f796cedd5f916, State: Running, Role: LEADER
I20260809 01:02:37.446106 20406 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } }
I20260809 01:02:37.446914 20402 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:02:37.448608 20408 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb7a947c18db4082b29f796cedd5f916. Latest consensus state: current_term: 3 leader_uuid: "fb7a947c18db4082b29f796cedd5f916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } } }
I20260809 01:02:37.449086 20408 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:37.449682 20407 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "fb7a947c18db4082b29f796cedd5f916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7a947c18db4082b29f796cedd5f916" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41471 } } }
I20260809 01:02:37.449983 20407 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7a947c18db4082b29f796cedd5f916 [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:37.460223 20412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:02:37.468751 20412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:02:37.470392 20412 catalog_manager.cc:1259] Loaded cluster ID: 919aede74558473eb07f04873c05e53a
I20260809 01:02:37.470535 20412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 01:02:37.472503 20412 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdf377729ad google::LogMessage::Fail() at ??:0
@ 0x7fdf3777588a google::LogMessage::SendToLog() at ??:0
@ 0x7fdf377724de google::LogMessage::Flush() at ??:0
@ 0x7fdf37773a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdf4d1a5512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
I20260809 01:02:37.501057 20427 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fdf4d1a2976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fdf4d22a610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fdf4d22a439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fdf3a3be345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fdf3a3c191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fdf3a3c1799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fdf3a39ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdf3d6766db start_thread at ??:0
@ 0x7fdf3660c61f clone at ??:0
W20260809 01:02:37.818171 20404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:37.818420 20404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:37.818511 20404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:37.825555 20404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:37.825667 20404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:37.825773 20404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:02:37.834892 20404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:35817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=37677
--tserver_master_addrs=127.4.82.190:41471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:37.836091 20404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:37.837255 20404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:37.849936 20446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:37.849942 20447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:37.850328 20449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:37.850993 20404 server_base.cc:1034] running on GCE node
I20260809 01:02:37.851593 20404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:37.852432 20404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:37.853652 20404 hybrid_clock.cc:648] HybridClock initialized: now 1786237357853610 us; error 51 us; skew 500 ppm
I20260809 01:02:37.853946 20404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:37 dist-test-slave-07c2 krb5kdc[19200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237357, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:37.857188 20404 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:02:37.858577 20404 webserver.cc:460] Webserver started at http://127.4.82.129:37677/ using document root <none> and password file <none>
I20260809 01:02:37.859160 20404 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:37.859267 20404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:37.864293 20404 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 01:02:37.866829 20456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:37.868109 20404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 01:02:37.868247 20404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "10a7ff25c6194e618d721e2597074ccb"
format_stamp: "Formatted at 2026-08-09 01:02:32 on dist-test-slave-07c2"
server_key: "bc0e5bad69ee34e73976ac362440aa69"
server_key_iv: "b617926b9e39cc1ee59ceb81ba4ff596"
server_key_version: "encryptionkey@0"
I20260809 01:02:37.869140 20404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:37.892441 20404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:37.896677 20404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:37.896898 20404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:37.897821 20404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:37.899183 20404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:37.899304 20404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:37.899446 20404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:37.899505 20404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:37.933280 20404 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:35817
I20260809 01:02:37.933370 20571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:35817 every 8 connection(s)
I20260809 01:02:37.934700 20404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:02:37.938910 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 20404
I20260809 01:02:37.939389 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:37.937537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:36493)
0809 01:02:37.938234 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:37.938482 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":273,"threads_started":1}
I20260809 01:02:37.939592 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:41017
--local_ip_for_outbound_sockets=127.4.82.130
--tserver_master_addrs=127.4.82.190:41471
--webserver_port=46269
--webserver_interface=127.4.82.130
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:37.940204 20572 heartbeater.cc:644] Failed to heartbeat to 127.4.82.190:41471 (0 consecutive failures): Network error: Failed to ping master at 127.4.82.190:41471: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
I20260809 01:02:37.941023 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:37.940733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:43497)
0809 01:02:37.940869 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:37.940931 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 01:02:37.941817 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:37.941546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:35381)
0809 01:02:37.941662 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:37.941724 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 01:02:37.942049 20572 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 01:02:38.268074 20575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:38.268316 20575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:38.268405 20575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:38.275444 20575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:38.275538 20575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:38.275650 20575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:02:38.284898 20575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:41017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=46269
--tserver_master_addrs=127.4.82.190:41471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:38.285982 20575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:38.287149 20575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:38.299850 20584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:38.299927 20582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:38.300115 20581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:38.300733 20575 server_base.cc:1034] running on GCE node
I20260809 01:02:38.301301 20575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:38.302150 20575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:38.303380 20575 hybrid_clock.cc:648] HybridClock initialized: now 1786237358303346 us; error 49 us; skew 500 ppm
I20260809 01:02:38.303690 20575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:38 dist-test-slave-07c2 krb5kdc[19200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237358, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:38.306613 20575 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:02:38.308038 20575 webserver.cc:460] Webserver started at http://127.4.82.130:46269/ using document root <none> and password file <none>
I20260809 01:02:38.308754 20575 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:38.308899 20575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:38.313795 20575 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 01:02:38.316323 20591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:38.317504 20575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 01:02:38.317684 20575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "11dd0a98f5de4e6fbc5b35ba5f6ea1f4"
format_stamp: "Formatted at 2026-08-09 01:02:32 on dist-test-slave-07c2"
server_key: "effc839db13e3fdde54dcf47d29f412d"
server_key_iv: "d75f7ebf24f547d2cb2e2695b838a0fa"
server_key_version: "encryptionkey@0"
I20260809 01:02:38.318627 20575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:38.353652 20575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:38.357786 20575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:38.358067 20575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:38.359033 20575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:38.360349 20575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:38.360471 20575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:38.360607 20575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:38.360695 20575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:38.392303 20575 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:41017
I20260809 01:02:38.392385 20709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:41017 every 8 connection(s)
I20260809 01:02:38.393671 20575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:02:38.395478 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 20575
I20260809 01:02:38.396085 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:38429
--local_ip_for_outbound_sockets=127.4.82.131
--tserver_master_addrs=127.4.82.190:41471
--webserver_port=34771
--webserver_interface=127.4.82.131
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:42631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:02:38.398860 20712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:38.396638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:41263)
0809 01:02:38.397484 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:38.397765 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":311,"threads_started":1}
W20260809 01:02:38.399544 20710 heartbeater.cc:644] Failed to heartbeat to 127.4.82.190:41471 (0 consecutive failures): Network error: Failed to ping master at 127.4.82.190:41471: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
I20260809 01:02:38.400447 20712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:38.400040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:39527)
0809 01:02:38.400218 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:38.400307 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 01:02:38.401391 20712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:38.400984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:45709)
0809 01:02:38.401143 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:38.401237 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 01:02:38.401638 20710 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 01:02:38.716518 20713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:38.716776 20713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:38.716867 20713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:38.723729 20713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:38.723822 20713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:38.723934 20713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:02:38.732944 20713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:38429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=34771
--tserver_master_addrs=127.4.82.190:41471
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:38.733971 20713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:38.735111 20713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:38.747915 20719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:38.747972 20718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:38.747999 20721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:38.748638 20713 server_base.cc:1034] running on GCE node
I20260809 01:02:38.749217 20713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:38.750070 20713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:38.751338 20713 hybrid_clock.cc:648] HybridClock initialized: now 1786237358751252 us; error 45 us; skew 500 ppm
I20260809 01:02:38.751644 20713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:38 dist-test-slave-07c2 krb5kdc[19200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237358, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:38.754534 20713 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:02:38.755906 20713 webserver.cc:460] Webserver started at http://127.4.82.131:34771/ using document root <none> and password file <none>
I20260809 01:02:38.756541 20713 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:38.756687 20713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:38.761420 20713 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 01:02:38.763861 20728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:38.765009 20713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 01:02:38.765188 20713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "da002cbbe93848e7a8188d112dc753c7"
format_stamp: "Formatted at 2026-08-09 01:02:33 on dist-test-slave-07c2"
server_key: "8a6f94006ebfeab36de78c031e3f4a62"
server_key_iv: "e2f296402b8b67d066a14628d8ed5f5b"
server_key_version: "encryptionkey@0"
I20260809 01:02:38.766106 20713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:38.794838 20713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:38.798434 20713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:38.798704 20713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:38.799639 20713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:38.800848 20713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:38.800979 20713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:38.801111 20713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:38.801210 20713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:38.833806 20713 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:38429
I20260809 01:02:38.833885 20848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:38429 every 8 connection(s)
I20260809 01:02:38.835187 20713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:02:38.839668 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:38.837932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:41557)
0809 01:02:38.838618 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:38.838821 (+ 203us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":276,"threads_started":1}
W20260809 01:02:38.840343 20850 heartbeater.cc:644] Failed to heartbeat to 127.4.82.190:41471 (0 consecutive failures): Network error: Failed to ping master at 127.4.82.190:41471: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
I20260809 01:02:38.841076 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:38.840802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:42559)
0809 01:02:38.840927 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:38.840988 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 01:02:38.841838 20852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:38.841565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:49455)
0809 01:02:38.841676 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:38.841742 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 01:02:38.842111 20850 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 01:02:38.842365 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 20713
I20260809 01:02:38.943176 20854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:38.942469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:60643)
0809 01:02:38.942889 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:38.942967 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":205,"threads_started":1}
I20260809 01:02:39.403002 20855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:39.402181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:34305)
0809 01:02:39.402671 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:39.402774 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":275,"threads_started":1}
I20260809 01:02:39.843349 20856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:39.842555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:38537)
0809 01:02:39.842982 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:39.843064 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":214,"threads_started":1}
I20260809 01:02:39.944893 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:39.944170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:53405)
0809 01:02:39.944593 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:39.944674 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":196,"threads_started":1}
I20260809 01:02:40.404814 20858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:40.404116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:47487)
0809 01:02:40.404562 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:40.404643 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":240,"threads_started":1}
I20260809 01:02:40.844830 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:40.844110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:39329)
0809 01:02:40.844553 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:40.844640 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":228,"threads_started":1}
I20260809 01:02:40.946630 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:40.945866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:60461)
0809 01:02:40.946333 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:40.946412 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":262,"threads_started":1}
I20260809 01:02:41.406215 20861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:41.405565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:36453)
0809 01:02:41.405985 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:41.406058 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":234,"threads_started":1}
I20260809 01:02:41.846356 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:41.845691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:44227)
0809 01:02:41.846110 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:41.846192 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":223,"threads_started":1}
I20260809 01:02:41.948210 20865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:41.947471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:38539)
0809 01:02:41.947952 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:41.948029 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":249,"threads_started":1}
I20260809 01:02:42.407825 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:42.407109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:59299)
0809 01:02:42.407577 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:42.407653 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":268,"threads_started":1}
I20260809 01:02:42.847967 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:42.847209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:49389)
0809 01:02:42.847686 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:42.847766 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":256,"threads_started":1}
I20260809 01:02:42.949792 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:42.949084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:49671)
0809 01:02:42.949538 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:42.949618 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":255,"threads_started":1}
I20260809 01:02:43.409526 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:43.408796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:56455)
0809 01:02:43.409278 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:43.409362 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":251,"threads_started":1}
I20260809 01:02:43.849596 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:43.848909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:56295)
0809 01:02:43.849356 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:43.849439 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":236,"threads_started":1}
I20260809 01:02:43.951534 20871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:43.950680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:53197)
0809 01:02:43.951271 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:43.951382 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":347,"threads_started":1}
I20260809 01:02:44.410988 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:44.410335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:36293)
0809 01:02:44.410748 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:44.410833 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":236,"threads_started":1}
I20260809 01:02:44.851135 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:44.850432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:46127)
0809 01:02:44.850893 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:44.850980 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":266,"threads_started":1}
I20260809 01:02:44.953116 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:44.952347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:60363)
0809 01:02:44.952826 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:44.952909 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":193,"threads_started":1}
I20260809 01:02:45.412621 20876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:45.411873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:44443)
0809 01:02:45.412337 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:45.412418 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":205,"threads_started":1}
I20260809 01:02:45.852692 20877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:45.851946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:36485)
0809 01:02:45.852420 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:45.852498 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":236,"threads_started":1}
I20260809 01:02:45.954855 20878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:45.954092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:60313)
0809 01:02:45.954561 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:45.954660 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":253,"threads_started":1}
I20260809 01:02:46.414137 20879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:46.413493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:51081)
0809 01:02:46.413903 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:46.413977 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":226,"threads_started":1}
I20260809 01:02:46.854286 20880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:46.853667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:50935)
0809 01:02:46.854058 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:46.854131 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":205,"threads_started":1}
I20260809 01:02:46.956390 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:46.955689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:33263)
0809 01:02:46.956136 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:46.956211 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":211,"threads_started":1}
I20260809 01:02:47.415791 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:47.415020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:57529)
0809 01:02:47.415519 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:47.415619 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":252,"threads_started":1}
I20260809 01:02:47.855960 20883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:47.855208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:39329)
0809 01:02:47.855698 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:47.855801 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":278,"threads_started":1}
I20260809 01:02:47.958029 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:47.957264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:33463)
0809 01:02:47.957752 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:47.957870 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":295,"threads_started":1}
I20260809 01:02:48.417335 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:48.416713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:39701)
0809 01:02:48.417103 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:48.417176 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":215,"threads_started":1}
I20260809 01:02:48.857718 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:48.856929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:41321)
0809 01:02:48.857476 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:48.857568 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":307,"threads_started":1}
I20260809 01:02:48.959551 20887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:48.958819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:44397)
0809 01:02:48.959233 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:48.959375 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":204,"threads_started":1}
I20260809 01:02:49.418821 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:49.418191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:42709)
0809 01:02:49.418576 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:49.418658 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":196,"threads_started":1}
I20260809 01:02:49.859517 20889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:49.858786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:52849)
0809 01:02:49.859209 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:49.859337 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":206,"threads_started":1}
I20260809 01:02:49.961112 20890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:49.960439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:36663)
0809 01:02:49.960885 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:49.960960 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":260,"threads_started":1}
I20260809 01:02:50.420512 20891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:50.419829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:38399)
0809 01:02:50.420269 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:50.420346 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":215,"threads_started":1}
I20260809 01:02:50.861055 20892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:50.860400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:39917)
0809 01:02:50.860839 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:50.860913 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":235,"threads_started":1}
I20260809 01:02:50.962775 20893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:50.962093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:50413)
0809 01:02:50.962499 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:50.962601 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":209,"threads_started":1}
I20260809 01:02:51.422241 20894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:51.421424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:42731)
0809 01:02:51.421957 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:51.422049 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":299,"threads_started":1}
I20260809 01:02:51.862566 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:51.861837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:33167)
0809 01:02:51.862297 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:51.862386 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":243,"threads_started":1}
I20260809 01:02:51.964337 20896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:51.963640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:40787)
0809 01:02:51.964062 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:51.964154 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":223,"threads_started":1}
I20260809 01:02:52.423750 20897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:52.423007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:52473)
0809 01:02:52.423483 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:52.423562 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":228,"threads_started":1}
I20260809 01:02:52.864138 20898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:52.863459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:53295)
0809 01:02:52.863876 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:52.863952 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":217,"threads_started":1}
I20260809 01:02:52.966051 20899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:52.965350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.129:52951)
0809 01:02:52.965776 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:52.965888 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":199,"threads_started":1}
I20260809 01:02:53.425420 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:53.424736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.130:52149)
0809 01:02:53.425151 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:53.425245 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":209,"threads_started":1}
I20260809 01:02:53.853587 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 20404
I20260809 01:02:53.865998 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:53.865229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:41471 (local address 127.4.82.131:59397)
0809 01:02:53.865727 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:53.865822 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:41471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":285,"threads_started":1}
I20260809 01:02:53.872399 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:53.854562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45680 (local address 127.4.82.129:35817)
0809 01:02:53.855262 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:53.855280 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:53.855348 (+ 68us) server_negotiation.cc:407] Connection header received
0809 01:02:53.857868 (+ 2520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:53.857878 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:53.858285 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:53.858636 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:53.860240 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:53.861481 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:53.862769 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:53.863471 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:53.863671 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:53.866422 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:53.866429 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:53.866442 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:53.866480 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:53.869220 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:53.869929 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:53.869942 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:53.869947 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:53.870102 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:53.870516 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:53.870519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:53.870521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:53.870967 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:53.871253 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:53.871745 (+ 492us) server_negotiation.cc:299] Negotiation successful
0809 01:02:53.872137 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":350,"threads_started":1}
I20260809 01:02:53.873704 20566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:53.958643 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 20404
I20260809 01:02:53.980294 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 20575
I20260809 01:02:53.997511 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:53.981295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44936 (local address 127.4.82.130:41017)
0809 01:02:53.981882 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:53.981896 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:53.981933 (+ 37us) server_negotiation.cc:407] Connection header received
0809 01:02:53.983633 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:53.983643 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:53.984074 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:53.984359 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:53.985766 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:53.986967 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:53.988301 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:53.988972 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:53.989102 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:53.991849 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:53.991857 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:53.991871 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:53.991918 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:53.994652 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:53.995328 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:53.995343 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:53.995351 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:53.995514 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:53.995845 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:53.995848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:53.995850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:53.996327 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:53.996559 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:53.996887 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 01:02:53.997204 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":287,"threads_started":1}
I20260809 01:02:53.998878 20704 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:54.083563 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 20575
I20260809 01:02:54.103668 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 20713
I20260809 01:02:54.121889 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:54.104753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58994 (local address 127.4.82.131:38429)
0809 01:02:54.105369 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:54.105383 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:54.105425 (+ 42us) server_negotiation.cc:407] Connection header received
0809 01:02:54.107030 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:54.107040 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:54.107515 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:54.107800 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:54.109077 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:54.110287 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:54.111577 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:54.112256 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:54.112456 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:54.116038 (+ 3582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:54.116045 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:54.116059 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:54.116094 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:54.118872 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:54.119634 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:54.119640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:54.119647 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:54.119838 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:54.120188 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:54.120191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:54.120193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:54.120671 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:54.120909 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:54.121265 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 01:02:54.121574 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":295,"threads_started":1}
I20260809 01:02:54.123217 20844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:54.206902 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 20713
2026-08-09T01:02:54Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24379 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:02:54 dist-test-slave-07c2 krb5kdc[20910](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:02:54 dist-test-slave-07c2 krb5kdc[20910](info): set up 2 sockets
Aug 09 01:02:54 dist-test-slave-07c2 krb5kdc[20910](info): commencing operation
krb5kdc: starting...
W20260809 01:02:55.332798 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:02:55 dist-test-slave-07c2 krb5kdc[20910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:02:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:02:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:02:55.588444 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34755
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:34755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:55.918726 20923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:55.918975 20923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:55.919130 20923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:55.926131 20923 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 01:02:55.926214 20923 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 01:02:55.926293 20923 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 01:02:55.926393 20923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:02:55.926481 20923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:55.926604 20923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:55.926692 20923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:02:55.926774 20923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:02:55.936043 20923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:34755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:55.937173 20923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:55.938407 20923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:55.949770 20928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:55.949884 20929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:55.949863 20931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:55.950469 20923 server_base.cc:1034] running on GCE node
I20260809 01:02:55.951161 20923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:55.952419 20923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:55.953655 20923 hybrid_clock.cc:648] HybridClock initialized: now 1786237375953601 us; error 30 us; skew 500 ppm
I20260809 01:02:55.953963 20923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:55 dist-test-slave-07c2 krb5kdc[20910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237375, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:55.960747 20923 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:02:55.962069 20923 webserver.cc:460] Webserver started at http://127.4.82.190:46449/ using document root <none> and password file <none>
I20260809 01:02:55.962718 20923 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:55.962843 20923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:55.963152 20923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:02:55.965934 20923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "c842fa5705564b6ab185134c3534ffa8"
format_stamp: "Formatted at 2026-08-09 01:02:55 on dist-test-slave-07c2"
server_key: "ac2a91f20bc804c09d9ab097859b4e9e"
server_key_iv: "6498dd7efd43125383cff29f86024560"
server_key_version: "encryptionkey@0"
I20260809 01:02:55.966701 20923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "c842fa5705564b6ab185134c3534ffa8"
format_stamp: "Formatted at 2026-08-09 01:02:55 on dist-test-slave-07c2"
server_key: "ac2a91f20bc804c09d9ab097859b4e9e"
server_key_iv: "6498dd7efd43125383cff29f86024560"
server_key_version: "encryptionkey@0"
I20260809 01:02:55.971675 20923 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 01:02:55.974876 20939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:55.976183 20923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:02:55.976364 20923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "c842fa5705564b6ab185134c3534ffa8"
format_stamp: "Formatted at 2026-08-09 01:02:55 on dist-test-slave-07c2"
server_key: "ac2a91f20bc804c09d9ab097859b4e9e"
server_key_iv: "6498dd7efd43125383cff29f86024560"
server_key_version: "encryptionkey@0"
I20260809 01:02:55.976572 20923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:55.993649 20923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:55.997283 20923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:55.997532 20923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:56.015600 20923 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:34755
I20260809 01:02:56.015622 20993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:34755 every 8 connection(s)
I20260809 01:02:56.016904 20923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:02:56.020437 20994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:56.023475 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 20923
I20260809 01:02:56.023712 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:02:56.024610 4426 external_mini_cluster.cc:1428] Setting key 8600bbd821e22eeab7b09abdafb164b4
I20260809 01:02:56.033115 20994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Bootstrap starting.
I20260809 01:02:56.036809 20994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:56.038115 20994 log.cc:824] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Log is configured to *not* fsync() on all Append() calls
Aug 09 01:02:56 dist-test-slave-07c2 krb5kdc[20910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:56.040930 20994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: No bootstrap required, opened a new log
I20260809 01:02:56.048010 20997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:56.026658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.4.82.190:34755)
0809 01:02:56.027462 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:56.027472 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:56.027510 (+ 38us) server_negotiation.cc:407] Connection header received
0809 01:02:56.029113 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:56.029138 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:56.029498 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:56.029814 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:56.031655 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:56.032889 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:56.034734 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:56.035452 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:56.035580 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:56.040652 (+ 5072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:56.040659 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:56.040672 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:56.040708 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:56.043989 (+ 3281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:56.044672 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:56.044679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:56.044684 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:56.044868 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:56.045262 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:56.045268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:56.045271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:56.045800 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:56.046105 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:56.046562 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 01:02:56.046998 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":342,"threads_started":1}
I20260809 01:02:56.048681 20994 raft_consensus.cc:348] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } }
I20260809 01:02:56.049036 20994 raft_consensus.cc:374] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:56.049153 20994 raft_consensus.cc:729] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c842fa5705564b6ab185134c3534ffa8, State: Initialized, Role: FOLLOWER
I20260809 01:02:56.049787 20994 consensus_queue.cc:260] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } }
I20260809 01:02:56.050031 20994 raft_consensus.cc:388] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:02:56.050196 20994 raft_consensus.cc:482] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:02:56.050382 20994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:56.052670 20994 raft_consensus.cc:504] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } }
I20260809 01:02:56.053072 20994 leader_election.cc:302] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c842fa5705564b6ab185134c3534ffa8; no voters:
I20260809 01:02:56.053625 20994 leader_election.cc:288] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:02:56.053781 21000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:02:56.054451 21000 raft_consensus.cc:686] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 1 LEADER]: Becoming Leader. State: Replica: c842fa5705564b6ab185134c3534ffa8, State: Running, Role: LEADER
I20260809 01:02:56.054991 21000 consensus_queue.cc:237] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } }
I20260809 01:02:56.055666 20994 sys_catalog.cc:564] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:02:56.057509 21002 sys_catalog.cc:455] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c842fa5705564b6ab185134c3534ffa8. Latest consensus state: current_term: 1 leader_uuid: "c842fa5705564b6ab185134c3534ffa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } } }
I20260809 01:02:56.057852 21002 sys_catalog.cc:458] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:56.058735 21005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:02:56.058684 21001 sys_catalog.cc:455] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c842fa5705564b6ab185134c3534ffa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } } }
I20260809 01:02:56.059594 21001 sys_catalog.cc:458] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: This master's current role is: LEADER
I20260809 01:02:56.065384 21005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:02:56.074355 21005 catalog_manager.cc:1347] Generated new cluster ID: fbb136614e4f4fc9911895cc99c8cd08
I20260809 01:02:56.074468 21005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:02:56.084508 21005 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:02:56.085686 21005 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:02:56.092042 21005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Generated new TSK 0
I20260809 01:02:56.092896 21005 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 01:02:56.162947 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34755
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:56.494901 21022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:56.495146 21022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:56.495236 21022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:56.502139 21022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:56.502238 21022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:56.502424 21022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:02:56.511523 21022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:56.512681 21022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:56.513912 21022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:56.526273 21031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:56.526330 21029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:56.526364 21028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:56.527531 21022 server_base.cc:1034] running on GCE node
I20260809 01:02:56.528049 21022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:56.528914 21022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:56.530107 21022 hybrid_clock.cc:648] HybridClock initialized: now 1786237376530069 us; error 36 us; skew 500 ppm
I20260809 01:02:56.530405 21022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:56 dist-test-slave-07c2 krb5kdc[20910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237376, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:56.533480 21022 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:02:56.534911 21022 webserver.cc:460] Webserver started at http://127.4.82.129:35905/ using document root <none> and password file <none>
I20260809 01:02:56.535650 21022 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:56.535807 21022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:56.536132 21022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:02:56.539049 21022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "6c57020af77c48fdadb7155d662472d7"
format_stamp: "Formatted at 2026-08-09 01:02:56 on dist-test-slave-07c2"
server_key: "3d367208faf39b8e19cbe9098b650ccb"
server_key_iv: "73ce5ae85a9ca51bf9c0e0c6d739fc3b"
server_key_version: "encryptionkey@0"
I20260809 01:02:56.539896 21022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "6c57020af77c48fdadb7155d662472d7"
format_stamp: "Formatted at 2026-08-09 01:02:56 on dist-test-slave-07c2"
server_key: "3d367208faf39b8e19cbe9098b650ccb"
server_key_iv: "73ce5ae85a9ca51bf9c0e0c6d739fc3b"
server_key_version: "encryptionkey@0"
I20260809 01:02:56.544806 21022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 01:02:56.548484 21038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:56.549960 21022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 01:02:56.550137 21022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "6c57020af77c48fdadb7155d662472d7"
format_stamp: "Formatted at 2026-08-09 01:02:56 on dist-test-slave-07c2"
server_key: "3d367208faf39b8e19cbe9098b650ccb"
server_key_iv: "73ce5ae85a9ca51bf9c0e0c6d739fc3b"
server_key_version: "encryptionkey@0"
I20260809 01:02:56.550338 21022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:56.567387 21022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:56.571228 21022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:56.571548 21022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:56.572479 21022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:56.573727 21022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:56.573797 21022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:56.573959 21022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:56.574016 21022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:56.607995 21022 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:41329
I20260809 01:02:56.608081 21154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:41329 every 8 connection(s)
I20260809 01:02:56.609262 21022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:02:56.610807 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 21022
I20260809 01:02:56.611049 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:02:56.611977 4426 external_mini_cluster.cc:1428] Setting key 171c5822d0d9b1a433e1c323a14f26e1
Aug 09 01:02:56 dist-test-slave-07c2 krb5kdc[20910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237376, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:56.630190 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:56.611765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:35221 (local address 127.4.82.190:34755)
0809 01:02:56.613121 (+ 1356us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:56.613131 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:56.613879 (+ 748us) server_negotiation.cc:407] Connection header received
0809 01:02:56.615754 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:56.615763 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:56.615879 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:56.615995 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:56.617337 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:56.618472 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:56.620022 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:56.620798 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:56.620955 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:56.624412 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:56.624420 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:56.624428 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:56.624473 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:56.626981 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:56.627920 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:56.627924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:56.627926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:56.628041 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:56.628573 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:56.628576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:56.628578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:56.628812 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:56.628919 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:56.629780 (+ 861us) server_negotiation.cc:299] Negotiation successful
0809 01:02:56.629938 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1185,"thread_start_us":1051,"threads_started":1}
I20260809 01:02:56.631632 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:56.612442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:35221)
0809 01:02:56.613729 (+ 1287us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:56.613774 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:56.615501 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:56.616158 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:56.616165 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:56.616656 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:56.617164 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:56.617181 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:56.618649 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:56.618652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:56.619879 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:56.619890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:56.621008 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:56.621013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:56.621199 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:56.621846 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:56.621872 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:56.624205 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:56.627127 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:56.627138 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:56.627155 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:56.627799 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:56.628170 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:56.628179 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:56.628185 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:56.628450 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:56.628941 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:56.628951 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:56.629406 (+ 455us) client_negotiation.cc:769] Sending connection context
0809 01:02:56.629696 (+ 290us) client_negotiation.cc:240] Negotiation successful
0809 01:02:56.630221 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"thread_start_us":784,"threads_started":1}
I20260809 01:02:56.633162 21155 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34755
I20260809 01:02:56.633436 21155 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:56.634096 21155 heartbeater.cc:511] Master 127.4.82.190:34755 requested a full tablet report, sending...
I20260809 01:02:56.636340 20959 ts_manager.cc:194] Registered new tserver with Master: 6c57020af77c48fdadb7155d662472d7 (127.4.82.129:41329)
I20260809 01:02:56.638577 20959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:35221
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 01:02:56.672767 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34755
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:56.994681 21161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:56.994916 21161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:56.995005 21161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:57.001976 21161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:57.002103 21161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:57.002292 21161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:02:57.011502 21161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:57.012710 21161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:57.013939 21161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:57.028086 21167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:57.028116 21169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:57.028926 21166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:57.029223 21161 server_base.cc:1034] running on GCE node
I20260809 01:02:57.029927 21161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:57.030946 21161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:57.032171 21161 hybrid_clock.cc:648] HybridClock initialized: now 1786237377032098 us; error 82 us; skew 500 ppm
I20260809 01:02:57.032483 21161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:57 dist-test-slave-07c2 krb5kdc[20910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237377, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:57.036036 21161 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:02:57.037412 21161 webserver.cc:460] Webserver started at http://127.4.82.130:38217/ using document root <none> and password file <none>
I20260809 01:02:57.038028 21161 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:57.038188 21161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:57.038494 21161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:02:57.041327 21161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "78f474c1c85f4ab089294cf4ea52db06"
format_stamp: "Formatted at 2026-08-09 01:02:57 on dist-test-slave-07c2"
server_key: "7ca9290d15e281714d577512e272ff65"
server_key_iv: "113fd0c356f20bb9dd771b5c76127b06"
server_key_version: "encryptionkey@0"
I20260809 01:02:57.042089 21161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "78f474c1c85f4ab089294cf4ea52db06"
format_stamp: "Formatted at 2026-08-09 01:02:57 on dist-test-slave-07c2"
server_key: "7ca9290d15e281714d577512e272ff65"
server_key_iv: "113fd0c356f20bb9dd771b5c76127b06"
server_key_version: "encryptionkey@0"
I20260809 01:02:57.047252 21161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 01:02:57.050519 21177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:57.051815 21161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:02:57.051981 21161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "78f474c1c85f4ab089294cf4ea52db06"
format_stamp: "Formatted at 2026-08-09 01:02:57 on dist-test-slave-07c2"
server_key: "7ca9290d15e281714d577512e272ff65"
server_key_iv: "113fd0c356f20bb9dd771b5c76127b06"
server_key_version: "encryptionkey@0"
I20260809 01:02:57.052186 21161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:57.062932 21161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:57.066617 21161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:57.066900 21161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:57.067899 21161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:57.069190 21161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:57.069262 21161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:57.069339 21161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:57.069382 21161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:57.101438 21161 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:44101
I20260809 01:02:57.101485 21293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:44101 every 8 connection(s)
I20260809 01:02:57.102653 21161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:02:57.108377 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 21161
I20260809 01:02:57.108594 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:02:57.109117 4426 external_mini_cluster.cc:1428] Setting key 568303273fc8ab5b677d5f38c858d54f
Aug 09 01:02:57 dist-test-slave-07c2 krb5kdc[20910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237377, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:57.122035 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.105140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:51329 (local address 127.4.82.190:34755)
0809 01:02:57.105358 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:57.105364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:57.106422 (+ 1058us) server_negotiation.cc:407] Connection header received
0809 01:02:57.108324 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:57.108332 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:57.108428 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:57.108564 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:57.110041 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.110867 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.112419 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.112852 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.112989 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.116283 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:57.116288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:57.116291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:57.116322 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:57.118878 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:57.119659 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:57.119663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:57.119666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:57.119796 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:57.120218 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:57.120227 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:57.120229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:57.120484 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:57.120645 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:57.121359 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 01:02:57.121629 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 01:02:57.122747 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.105582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:51329)
0809 01:02:57.106257 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:57.106301 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:57.107967 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:57.108828 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:57.108836 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:57.109374 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:57.109891 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.109907 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.111012 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.111017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.112264 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.112275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.113071 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.113077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.113299 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.113969 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:57.113989 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:57.116066 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:57.119016 (+ 2950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:57.119024 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:57.119038 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:57.119529 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:57.119911 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:57.119916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:57.119920 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:57.120097 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:57.120621 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:57.120628 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:57.120934 (+ 306us) client_negotiation.cc:769] Sending connection context
0809 01:02:57.121195 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 01:02:57.121626 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":280,"threads_started":1}
I20260809 01:02:57.124717 21294 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34755
I20260809 01:02:57.125099 21294 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:57.125958 21294 heartbeater.cc:511] Master 127.4.82.190:34755 requested a full tablet report, sending...
I20260809 01:02:57.127632 20959 ts_manager.cc:194] Registered new tserver with Master: 78f474c1c85f4ab089294cf4ea52db06 (127.4.82.130:44101)
I20260809 01:02:57.128911 20959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:51329
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 01:02:57.170753 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34755
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:02:57.494273 21299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:02:57.494519 21299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:02:57.494741 21299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:02:57.501722 21299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:02:57.501839 21299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:02:57.502027 21299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:02:57.511735 21299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:02:57.512995 21299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:02:57.514261 21299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:02:57.526746 21305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:57.526827 21304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:02:57.526799 21307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:02:57.527989 21299 server_base.cc:1034] running on GCE node
I20260809 01:02:57.528592 21299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:02:57.529470 21299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:02:57.530705 21299 hybrid_clock.cc:648] HybridClock initialized: now 1786237377530657 us; error 39 us; skew 500 ppm
I20260809 01:02:57.531026 21299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:02:57 dist-test-slave-07c2 krb5kdc[20910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237377, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:02:57.534178 21299 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:02:57.535688 21299 webserver.cc:460] Webserver started at http://127.4.82.131:36825/ using document root <none> and password file <none>
I20260809 01:02:57.536334 21299 fs_manager.cc:359] Metadata directory not provided
I20260809 01:02:57.536479 21299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:02:57.536789 21299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:02:57.539664 21299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "16279b6f436543a8905c001ad14f4309"
format_stamp: "Formatted at 2026-08-09 01:02:57 on dist-test-slave-07c2"
server_key: "884e9ae10c1aa8c356830ccec0583af7"
server_key_iv: "f8977ec604eeaacc242a7966fe8cd219"
server_key_version: "encryptionkey@0"
I20260809 01:02:57.540416 21299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "16279b6f436543a8905c001ad14f4309"
format_stamp: "Formatted at 2026-08-09 01:02:57 on dist-test-slave-07c2"
server_key: "884e9ae10c1aa8c356830ccec0583af7"
server_key_iv: "f8977ec604eeaacc242a7966fe8cd219"
server_key_version: "encryptionkey@0"
I20260809 01:02:57.545291 21299 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 01:02:57.548673 21315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:57.550169 21299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 01:02:57.550343 21299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "16279b6f436543a8905c001ad14f4309"
format_stamp: "Formatted at 2026-08-09 01:02:57 on dist-test-slave-07c2"
server_key: "884e9ae10c1aa8c356830ccec0583af7"
server_key_iv: "f8977ec604eeaacc242a7966fe8cd219"
server_key_version: "encryptionkey@0"
I20260809 01:02:57.550537 21299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:02:57.570595 21299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:02:57.575978 21299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:02:57.576303 21299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:02:57.578079 21299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:02:57.579855 21299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:02:57.579944 21299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:02:57.580078 21299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:02:57.580188 21299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 01:02:57.613624 21299 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:39695
I20260809 01:02:57.613646 21430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:39695 every 8 connection(s)
I20260809 01:02:57.615182 21299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:02:57.625075 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 21299
I20260809 01:02:57.625272 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:02:57.625871 4426 external_mini_cluster.cc:1428] Setting key a264b0cb263082e97ca926e4ea7210dd
Aug 09 01:02:57 dist-test-slave-07c2 krb5kdc[20910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237377, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:02:57.634783 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.617770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:38035 (local address 127.4.82.190:34755)
0809 01:02:57.617948 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:57.617953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:57.618979 (+ 1026us) server_negotiation.cc:407] Connection header received
0809 01:02:57.620786 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:57.620793 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:57.620899 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:57.621004 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:57.622445 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.623206 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.624794 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.625278 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.625425 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.629734 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:57.629738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:57.629742 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:57.629772 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:57.632363 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:57.632997 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:57.633001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:57.633003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:57.633123 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:57.633460 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:57.633462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:57.633464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:57.633712 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:57.633817 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:57.634367 (+ 550us) server_negotiation.cc:299] Negotiation successful
0809 01:02:57.634537 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 01:02:57.635762 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.618146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:38035)
0809 01:02:57.618818 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:57.618859 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:57.620501 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:57.621228 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:57.621235 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:57.621726 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:02:57.622274 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.622290 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.623385 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.623395 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.624608 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.624619 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.625558 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.625567 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.625852 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.626566 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:02:57.626592 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:02:57.629557 (+ 2965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:02:57.632494 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:57.632499 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:57.632508 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:57.632896 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:57.633209 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:02:57.633212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:02:57.633214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:02:57.633367 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:02:57.633809 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:02:57.633814 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:02:57.634073 (+ 259us) client_negotiation.cc:769] Sending connection context
0809 01:02:57.634268 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 01:02:57.634614 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":270,"threads_started":1}
I20260809 01:02:57.637044 21431 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34755
I20260809 01:02:57.637356 21431 heartbeater.cc:464] Registering TS with master...
I20260809 01:02:57.638008 21431 heartbeater.cc:511] Master 127.4.82.190:34755 requested a full tablet report, sending...
I20260809 01:02:57.639417 20959 ts_manager.cc:194] Registered new tserver with Master: 16279b6f436543a8905c001ad14f4309 (127.4.82.131:39695)
I20260809 01:02:57.640230 20959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:38035
I20260809 01:02:57.641306 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:02:57.641467 21155 heartbeater.cc:503] Master 127.4.82.190:34755 was elected leader, sending a full tablet report...
I20260809 01:02:57.658269 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.646159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34416 (local address 127.4.82.190:34755)
0809 01:02:57.646333 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:57.646341 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:57.646786 (+ 445us) server_negotiation.cc:407] Connection header received
0809 01:02:57.646889 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:57.646893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:57.646945 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:57.647018 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:57.648471 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.649176 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.650472 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.650882 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.650966 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.653110 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:57.653114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:57.653117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:57.653144 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:57.655970 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:57.656604 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:57.656608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:57.656610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:57.656740 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:57.657094 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:57.657096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:57.657098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:57.657430 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:57.657606 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:57.657852 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 01:02:57.657995 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 01:02:57.662078 20959 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:02:57.668874 20959 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:02:57.701769 21446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.680642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51918 (local address 127.4.82.131:39695)
0809 01:02:57.693452 (+ 12810us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:57.693460 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:57.693489 (+ 29us) server_negotiation.cc:407] Connection header received
0809 01:02:57.693583 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:57.693592 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:57.693857 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:57.694440 (+ 583us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:57.695155 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.697321 (+ 2166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.699229 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.700589 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.700752 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.700991 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:57.701223 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 01:02:57.701385 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12647,"thread_start_us":12825,"threads_started":1}
I20260809 01:02:57.701889 21445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.680273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:39695 (local address 127.0.0.1:51918)
0809 01:02:57.681319 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:57.681375 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:57.681570 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:57.694084 (+ 12514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:57.694087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:57.694125 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:57.694911 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.694927 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.697457 (+ 2530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.697464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.699066 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.699082 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.700780 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.700786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.700977 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.701033 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 01:02:57.701183 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 01:02:57.701386 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":80,"spinlock_wait_cycles":4608,"thread_start_us":12644,"threads_started":1}
I20260809 01:02:57.702737 21448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.682249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:44101 (local address 127.0.0.1:49490)
0809 01:02:57.693197 (+ 10948us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:57.693224 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:57.693360 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:57.694069 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:57.694076 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:57.694118 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:57.694699 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.694712 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.696663 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.696669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.699540 (+ 2871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.699552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.701680 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.701696 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.701968 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.702113 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 01:02:57.702200 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 01:02:57.702335 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10820,"mutex_wait_us":343,"spinlock_wait_cycles":768,"thread_start_us":10675,"threads_started":1}
I20260809 01:02:57.702781 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.680983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48528 (local address 127.4.82.129:41329)
0809 01:02:57.682500 (+ 1517us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:57.682512 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:57.692981 (+ 10469us) server_negotiation.cc:407] Connection header received
0809 01:02:57.693945 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:57.693955 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:57.694168 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:57.694331 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:57.695396 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.696663 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.699742 (+ 3079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.701176 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.701320 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.701532 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:57.702280 (+ 748us) server_negotiation.cc:299] Negotiation successful
0809 01:02:57.702445 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1325,"thread_start_us":1173,"threads_started":1}
I20260809 01:02:57.703529 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.682848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49490 (local address 127.4.82.130:44101)
0809 01:02:57.692898 (+ 10050us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:57.692907 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:57.693311 (+ 404us) server_negotiation.cc:407] Connection header received
0809 01:02:57.693531 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:57.693539 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:57.693815 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:57.694033 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:57.694883 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.696523 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.699723 (+ 3200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.701176 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.701321 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.701494 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:57.702980 (+ 1486us) server_negotiation.cc:299] Negotiation successful
0809 01:02:57.703144 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9881,"thread_start_us":9748,"threads_started":1}
I20260809 01:02:57.703559 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.680481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:48528)
0809 01:02:57.692835 (+ 12354us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:57.692857 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:57.693721 (+ 864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:57.694674 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:57.694680 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:57.694713 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:57.695212 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.695226 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.696866 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.696871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.699538 (+ 2667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.699552 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.701711 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.701724 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.701970 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.702111 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 01:02:57.703016 (+ 905us) client_negotiation.cc:240] Negotiation successful
0809 01:02:57.703143 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12171,"mutex_wait_us":216,"thread_start_us":259,"threads_started":1}
I20260809 01:02:57.706310 21226 tablet_service.cc:1467] Processing CreateTablet for tablet 75cd3b0a6b82436580247bbf15afade2 (DEFAULT_TABLE table=test-table [id=702eb87b38c74f55a2990cefa625fdf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:02:57.706305 21364 tablet_service.cc:1467] Processing CreateTablet for tablet 75cd3b0a6b82436580247bbf15afade2 (DEFAULT_TABLE table=test-table [id=702eb87b38c74f55a2990cefa625fdf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:02:57.706910 21088 tablet_service.cc:1467] Processing CreateTablet for tablet 75cd3b0a6b82436580247bbf15afade2 (DEFAULT_TABLE table=test-table [id=702eb87b38c74f55a2990cefa625fdf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:02:57.707616 21226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75cd3b0a6b82436580247bbf15afade2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:57.707728 21364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75cd3b0a6b82436580247bbf15afade2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:57.708251 21088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75cd3b0a6b82436580247bbf15afade2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:02:57.718371 21450 tablet_bootstrap.cc:492] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06: Bootstrap starting.
I20260809 01:02:57.721927 21451 tablet_bootstrap.cc:492] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: Bootstrap starting.
I20260809 01:02:57.723037 21450 tablet_bootstrap.cc:654] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:57.724087 21452 tablet_bootstrap.cc:492] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309: Bootstrap starting.
I20260809 01:02:57.724424 21450 log.cc:824] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:57.725618 21451 tablet_bootstrap.cc:654] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:57.726854 21451 log.cc:824] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:57.727116 21452 tablet_bootstrap.cc:654] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309: Neither blocks nor log segments found. Creating new log.
I20260809 01:02:57.727738 21450 tablet_bootstrap.cc:492] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06: No bootstrap required, opened a new log
I20260809 01:02:57.728003 21450 ts_tablet_manager.cc:1397] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260809 01:02:57.728350 21452 log.cc:824] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309: Log is configured to *not* fsync() on all Append() calls
I20260809 01:02:57.729861 21451 tablet_bootstrap.cc:492] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: No bootstrap required, opened a new log
I20260809 01:02:57.730154 21451 ts_tablet_manager.cc:1397] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260809 01:02:57.731025 21452 tablet_bootstrap.cc:492] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309: No bootstrap required, opened a new log
I20260809 01:02:57.731271 21452 ts_tablet_manager.cc:1397] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260809 01:02:57.737879 21452 raft_consensus.cc:348] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } }
I20260809 01:02:57.738209 21452 raft_consensus.cc:374] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:57.738329 21452 raft_consensus.cc:729] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16279b6f436543a8905c001ad14f4309, State: Initialized, Role: FOLLOWER
I20260809 01:02:57.738355 21450 raft_consensus.cc:348] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } }
I20260809 01:02:57.738840 21450 raft_consensus.cc:374] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:57.738942 21450 raft_consensus.cc:729] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78f474c1c85f4ab089294cf4ea52db06, State: Initialized, Role: FOLLOWER
I20260809 01:02:57.738986 21452 consensus_queue.cc:260] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } }
I20260809 01:02:57.739495 21450 consensus_queue.cc:260] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } }
I20260809 01:02:57.740268 21431 heartbeater.cc:503] Master 127.4.82.190:34755 was elected leader, sending a full tablet report...
I20260809 01:02:57.740866 21452 ts_tablet_manager.cc:1428] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309: Time spent starting tablet: real 0.009s user 0.007s sys 0.001s
I20260809 01:02:57.741483 21451 raft_consensus.cc:348] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } }
I20260809 01:02:57.741828 21451 raft_consensus.cc:374] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:02:57.741957 21451 raft_consensus.cc:729] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c57020af77c48fdadb7155d662472d7, State: Initialized, Role: FOLLOWER
I20260809 01:02:57.742522 21451 consensus_queue.cc:260] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } }
I20260809 01:02:57.744264 21451 ts_tablet_manager.cc:1428] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260809 01:02:57.745235 21294 heartbeater.cc:503] Master 127.4.82.190:34755 was elected leader, sending a full tablet report...
I20260809 01:02:57.745887 21450 ts_tablet_manager.cc:1428] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06: Time spent starting tablet: real 0.018s user 0.008s sys 0.009s
I20260809 01:02:57.833168 21457 raft_consensus.cc:482] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:02:57.833480 21457 raft_consensus.cc:504] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } }
I20260809 01:02:57.835086 21457 leader_election.cc:288] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16279b6f436543a8905c001ad14f4309 (127.4.82.131:39695), 6c57020af77c48fdadb7155d662472d7 (127.4.82.129:41329)
I20260809 01:02:57.842191 21446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.835430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:48141 (local address 127.4.82.131:39695)
0809 01:02:57.835641 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:57.835647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:57.836773 (+ 1126us) server_negotiation.cc:407] Connection header received
0809 01:02:57.836963 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:57.836968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:57.837037 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:57.837142 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:57.837721 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.838480 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.840464 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.841253 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.841422 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.841498 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:57.841796 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 01:02:57.841924 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 01:02:57.842216 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.835605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:55543 (local address 127.4.82.129:41329)
0809 01:02:57.835836 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:57.835846 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:57.836742 (+ 896us) server_negotiation.cc:407] Connection header received
0809 01:02:57.836958 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:57.836964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:57.837036 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:57.837147 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:02:57.837822 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.838522 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.840441 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.841213 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.841388 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.841468 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:57.841812 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 01:02:57.841937 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 01:02:57.842362 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.835582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:39695 (local address 127.4.82.130:48141)
0809 01:02:57.836660 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:57.836675 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:57.836832 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:57.837157 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:57.837161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:57.837183 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:57.837565 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.837573 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.838602 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.838609 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.840321 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.840332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.841477 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.841483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.841637 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.841671 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 01:02:57.841771 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 01:02:57.841924 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"mutex_wait_us":97,"spinlock_wait_cycles":29568,"thread_start_us":580,"threads_started":1}
I20260809 01:02:57.842420 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.835534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.4.82.130:55543)
0809 01:02:57.836626 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:57.836642 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:02:57.836810 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:02:57.837162 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:02:57.837167 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:02:57.837252 (+ 85us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:02:57.837678 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.837691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.838661 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.838666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.840293 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:02:57.840307 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.841442 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.841449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:02:57.841625 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.841670 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 01:02:57.841775 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 01:02:57.841934 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":60,"spinlock_wait_cycles":24448,"thread_start_us":488,"threads_started":1}
I20260809 01:02:57.843605 21109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75cd3b0a6b82436580247bbf15afade2" candidate_uuid: "78f474c1c85f4ab089294cf4ea52db06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c57020af77c48fdadb7155d662472d7" is_pre_election: true
I20260809 01:02:57.843911 21385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75cd3b0a6b82436580247bbf15afade2" candidate_uuid: "78f474c1c85f4ab089294cf4ea52db06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16279b6f436543a8905c001ad14f4309" is_pre_election: true
I20260809 01:02:57.844121 21109 raft_consensus.cc:2454] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78f474c1c85f4ab089294cf4ea52db06 in term 0.
I20260809 01:02:57.844471 21385 raft_consensus.cc:2454] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78f474c1c85f4ab089294cf4ea52db06 in term 0.
I20260809 01:02:57.844761 21178 leader_election.cc:302] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c57020af77c48fdadb7155d662472d7, 78f474c1c85f4ab089294cf4ea52db06; no voters:
I20260809 01:02:57.845121 21457 raft_consensus.cc:2781] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:02:57.845268 21457 raft_consensus.cc:482] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:02:57.845371 21457 raft_consensus.cc:3037] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:57.847579 21457 raft_consensus.cc:504] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } }
I20260809 01:02:57.848121 21457 leader_election.cc:288] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [CANDIDATE]: Term 1 election: Requested vote from peers 16279b6f436543a8905c001ad14f4309 (127.4.82.131:39695), 6c57020af77c48fdadb7155d662472d7 (127.4.82.129:41329)
I20260809 01:02:57.848559 21385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75cd3b0a6b82436580247bbf15afade2" candidate_uuid: "78f474c1c85f4ab089294cf4ea52db06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16279b6f436543a8905c001ad14f4309"
I20260809 01:02:57.848666 21109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75cd3b0a6b82436580247bbf15afade2" candidate_uuid: "78f474c1c85f4ab089294cf4ea52db06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c57020af77c48fdadb7155d662472d7"
I20260809 01:02:57.848796 21385 raft_consensus.cc:3037] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:57.848922 21109 raft_consensus.cc:3037] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:02:57.850986 21385 raft_consensus.cc:2454] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78f474c1c85f4ab089294cf4ea52db06 in term 1.
I20260809 01:02:57.851107 21109 raft_consensus.cc:2454] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78f474c1c85f4ab089294cf4ea52db06 in term 1.
I20260809 01:02:57.851397 21181 leader_election.cc:302] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16279b6f436543a8905c001ad14f4309, 78f474c1c85f4ab089294cf4ea52db06; no voters:
I20260809 01:02:57.851720 21457 raft_consensus.cc:2781] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:02:57.852159 21457 raft_consensus.cc:686] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [term 1 LEADER]: Becoming Leader. State: Replica: 78f474c1c85f4ab089294cf4ea52db06, State: Running, Role: LEADER
I20260809 01:02:57.852602 21457 consensus_queue.cc:237] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } }
W20260809 01:02:57.854537 21295 tablet.cc:2369] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:02:57.857983 20957 catalog_manager.cc:5495] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78f474c1c85f4ab089294cf4ea52db06 (127.4.82.130). New cstate: current_term: 1 leader_uuid: "78f474c1c85f4ab089294cf4ea52db06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } health_report { overall_health: HEALTHY } } }
W20260809 01:02:57.861697 21156 tablet.cc:2369] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 01:02:57.866719 21432 tablet.cc:2369] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:02:57.893044 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.888132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49504 (local address 127.4.82.130:44101)
0809 01:02:57.888326 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:57.888330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:57.888438 (+ 108us) server_negotiation.cc:407] Connection header received
0809 01:02:57.888624 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:57.888628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:57.888698 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:57.888825 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:02:57.889161 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.889869 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.891161 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.891580 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.891671 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.892109 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:02:57.892552 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:02:57.892660 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:57.892765 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 01:02:57.892825 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 01:02:57.900866 21385 raft_consensus.cc:1261] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309 [term 1 FOLLOWER]: Refusing update from remote peer 78f474c1c85f4ab089294cf4ea52db06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:02:57.900868 21109 raft_consensus.cc:1261] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7 [term 1 FOLLOWER]: Refusing update from remote peer 78f474c1c85f4ab089294cf4ea52db06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:02:57.901906 21462 consensus_queue.cc:1035] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:02:57.902283 21457 consensus_queue.cc:1035] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:02:57.912899 21465 mvcc.cc:205] Tried to move back new op lower bound from 7316428299871113216 to 7316428299688710144. Current Snapshot: MvccSnapshot[applied={T|T < 7316428299871113216}]
I20260809 01:02:57.925839 20957 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:02:57.926131 20957 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:02:57.929733 20957 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Sending DeleteTablet for 3 replicas of tablet 75cd3b0a6b82436580247bbf15afade2
I20260809 01:02:57.930729 21364 tablet_service.cc:1514] Processing DeleteTablet for tablet 75cd3b0a6b82436580247bbf15afade2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:02:57 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:51918
I20260809 01:02:57.931025 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 21022
I20260809 01:02:57.931134 21088 tablet_service.cc:1514] Processing DeleteTablet for tablet 75cd3b0a6b82436580247bbf15afade2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:02:57 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:48528
I20260809 01:02:57.931811 21226 tablet_service.cc:1514] Processing DeleteTablet for tablet 75cd3b0a6b82436580247bbf15afade2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:02:57 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:49490
I20260809 01:02:57.946621 21476 tablet_replica.cc:330] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309: stopping tablet replica
I20260809 01:02:57.946903 21479 tablet_replica.cc:330] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06: stopping tablet replica
I20260809 01:02:57.947224 21476 raft_consensus.cc:2229] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:02:57.947835 21476 raft_consensus.cc:2258] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:02:57.947357 21479 raft_consensus.cc:2229] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:02:57.948571 21479 raft_consensus.cc:2258] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:02:57.947878 21477 tablet_replica.cc:330] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: stopping tablet replica
I20260809 01:02:57.949074 21477 raft_consensus.cc:2229] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:02:57.949762 21477 raft_consensus.cc:2258] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:02:57.950186 21476 ts_tablet_manager.cc:1905] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:02:57.952282 21479 ts_tablet_manager.cc:1905] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:02:57.955987 21477 ts_tablet_manager.cc:1905] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 01:02:57 dist-test-slave-07c2 krb5kdc[20910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 01:02:57.957489 21476 ts_tablet_manager.cc:1918] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:02:57.957650 21476 log.cc:1196] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/75cd3b0a6b82436580247bbf15afade2
I20260809 01:02:57.958216 21476 ts_tablet_manager.cc:1939] T 75cd3b0a6b82436580247bbf15afade2 P 16279b6f436543a8905c001ad14f4309: Deleting consensus metadata
I20260809 01:02:57.960126 20943 catalog_manager.cc:4841] TS 16279b6f436543a8905c001ad14f4309 (127.4.82.131:39695): tablet 75cd3b0a6b82436580247bbf15afade2 (table test-table [id=702eb87b38c74f55a2990cefa625fdf5]) successfully deleted
I20260809 01:02:57.964951 21447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:57.932183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48538 (local address 127.4.82.129:41329)
0809 01:02:57.932345 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:57.932351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:57.943152 (+ 10801us) server_negotiation.cc:407] Connection header received
0809 01:02:57.943458 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:57.943466 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:57.943587 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:57.943667 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:57.947754 (+ 4087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.948964 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.951025 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:57.951740 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:57.951838 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:57.957933 (+ 6095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:57.957949 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:57.957966 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:57.958031 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:57.961869 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:57.962631 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:57.962639 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:57.962642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:57.962781 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:57.963156 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:57.963161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:57.963164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:57.963672 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:57.963999 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:57.964351 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 01:02:57.964574 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 01:02:57.965575 21479 ts_tablet_manager.cc:1918] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:02:57.965615 21150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:57.965749 21479 log.cc:1196] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/75cd3b0a6b82436580247bbf15afade2
I20260809 01:02:57.966310 21479 ts_tablet_manager.cc:1939] T 75cd3b0a6b82436580247bbf15afade2 P 78f474c1c85f4ab089294cf4ea52db06: Deleting consensus metadata
I20260809 01:02:57.968077 20940 catalog_manager.cc:4841] TS 78f474c1c85f4ab089294cf4ea52db06 (127.4.82.130:44101): tablet 75cd3b0a6b82436580247bbf15afade2 (table test-table [id=702eb87b38c74f55a2990cefa625fdf5]) successfully deleted
I20260809 01:02:58.054800 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 21022
I20260809 01:02:58.075390 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 21161
W20260809 01:02:58.075362 20940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.82.129:41329 (error 108)
I20260809 01:02:58.077488 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:58.076999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:48548)
0809 01:02:58.077184 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:58.077318 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 01:02:58.077740 20940 catalog_manager.cc:4584] TS 6c57020af77c48fdadb7155d662472d7 (127.4.82.129:41329): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 75cd3b0a6b82436580247bbf15afade2: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Aug 09 01:02:58 dist-test-slave-07c2 krb5kdc[20910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 01:02:58.092803 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:58.076530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49508 (local address 127.4.82.130:44101)
0809 01:02:58.076725 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:58.076731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:58.076848 (+ 117us) server_negotiation.cc:407] Connection header received
0809 01:02:58.077031 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:58.077037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:58.077120 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:58.077271 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:58.079048 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:58.080016 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:58.081400 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:58.081823 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:58.081958 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:58.086606 (+ 4648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:58.086617 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:58.086630 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:58.086665 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:58.089934 (+ 3269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:58.090669 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:58.090676 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:58.090678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:58.090817 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:58.091197 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:58.091203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:58.091206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:58.091585 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:58.091937 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:58.092185 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 01:02:58.092393 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 01:02:58.093505 21288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:58.131709 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:58.131073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:48554)
0809 01:02:58.131450 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:58.131555 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
I20260809 01:02:58.132769 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:58.132366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:48562)
0809 01:02:58.132512 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:58.132612 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 01:02:58.193598 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:58.193037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:48572)
0809 01:02:58.193353 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:58.193438 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
I20260809 01:02:58.194558 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:58.194229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:48584)
0809 01:02:58.194376 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:58.194448 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 01:02:58.211149 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 21161
I20260809 01:02:58.231351 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 21299
Aug 09 01:02:58 dist-test-slave-07c2 krb5kdc[20910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 01:02:58.246701 21446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:58.232114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51932 (local address 127.4.82.131:39695)
0809 01:02:58.232309 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 01:02:58.232313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:02:58.232331 (+ 18us) server_negotiation.cc:407] Connection header received
0809 01:02:58.232478 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:02:58.232482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:02:58.232554 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:02:58.232635 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:02:58.234029 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:58.234745 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:58.236193 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:02:58.236578 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:02:58.236672 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:02:58.241014 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:02:58.241028 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:02:58.241043 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:02:58.241075 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:02:58.244164 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:58.244928 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:58.244935 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:58.244937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:58.245081 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:02:58.245402 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:02:58.245405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:02:58.245407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:02:58.245747 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:02:58.245959 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 01:02:58.246205 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 01:02:58.246361 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 01:02:58.247429 21426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:58.275462 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:58.274978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:48596)
0809 01:02:58.275173 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:58.275263 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 01:02:58.276494 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:58.276097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:48598)
0809 01:02:58.276303 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:58.276375 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260809 01:02:58.335816 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 21299
I20260809 01:02:58.354231 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 20923
I20260809 01:02:58.355425 20989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:02:58.467092 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:02:58.466047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:51086)
0809 01:02:58.466612 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0809 01:02:58.466898 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215}
I20260809 01:02:58.467633 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 20923
I20260809 01:03:18.488441 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34755
--webserver_interface=127.4.82.190
--webserver_port=46449
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:34755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:03:18.816176 21485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:03:18.816412 21485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:03:18.816574 21485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:03:18.823503 21485 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 01:03:18.823586 21485 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 01:03:18.823685 21485 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 01:03:18.823782 21485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:03:18.823875 21485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:03:18.823989 21485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:03:18.824075 21485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:03:18.824150 21485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:03:18.833477 21485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:34755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=46449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:03:18.834628 21485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:03:18.835919 21485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:03:18.859727 21491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:18.859782 21492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:18.859757 21494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:03:18.861006 21485 server_base.cc:1034] running on GCE node
I20260809 01:03:18.861894 21485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:03:18.863372 21485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:03:18.864620 21485 hybrid_clock.cc:648] HybridClock initialized: now 1786237398864551 us; error 42 us; skew 500 ppm
I20260809 01:03:18.865073 21485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:03:18 dist-test-slave-07c2 krb5kdc[20910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237398, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:03:18.871860 21485 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:03:18.873328 21485 webserver.cc:460] Webserver started at http://127.4.82.190:46449/ using document root <none> and password file <none>
I20260809 01:03:18.874019 21485 fs_manager.cc:359] Metadata directory not provided
I20260809 01:03:18.874125 21485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:03:18.879985 21485 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 01:03:18.882654 21501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:18.884009 21485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:03:18.884196 21485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "c842fa5705564b6ab185134c3534ffa8"
format_stamp: "Formatted at 2026-08-09 01:02:55 on dist-test-slave-07c2"
server_key: "ac2a91f20bc804c09d9ab097859b4e9e"
server_key_iv: "6498dd7efd43125383cff29f86024560"
server_key_version: "encryptionkey@0"
I20260809 01:03:18.885273 21485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:03:18.912171 21485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:03:18.916167 21485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:03:18.916452 21485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:03:18.934986 21485 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:34755
I20260809 01:03:18.935027 21554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:34755 every 8 connection(s)
I20260809 01:03:18.936507 21485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:03:18.943056 21555 sys_catalog.cc:263] Verifying existing consensus state
I20260809 01:03:18.946184 21555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Bootstrap starting.
I20260809 01:03:18.946623 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 21485
I20260809 01:03:18.947482 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:41329
--local_ip_for_outbound_sockets=127.4.82.129
--tserver_master_addrs=127.4.82.190:34755
--webserver_port=35905
--webserver_interface=127.4.82.129
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:03:18.964994 21555 log.cc:824] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Log is configured to *not* fsync() on all Append() calls
I20260809 01:03:18.976972 21555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 01:03:18.977486 21555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Bootstrap complete.
I20260809 01:03:18.986218 21555 raft_consensus.cc:348] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } }
I20260809 01:03:18.987020 21555 raft_consensus.cc:729] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c842fa5705564b6ab185134c3534ffa8, State: Initialized, Role: FOLLOWER
I20260809 01:03:18.987684 21555 consensus_queue.cc:260] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } }
I20260809 01:03:18.987938 21555 raft_consensus.cc:388] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:03:18.988073 21555 raft_consensus.cc:482] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:03:18.988265 21555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 1 FOLLOWER]: Advancing to term 2
I20260809 01:03:18.992300 21555 raft_consensus.cc:504] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } }
I20260809 01:03:18.992808 21555 leader_election.cc:302] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c842fa5705564b6ab185134c3534ffa8; no voters:
I20260809 01:03:18.993444 21555 leader_election.cc:288] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 01:03:18.993577 21559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 01:03:18.994838 21559 raft_consensus.cc:686] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 2 LEADER]: Becoming Leader. State: Replica: c842fa5705564b6ab185134c3534ffa8, State: Running, Role: LEADER
I20260809 01:03:18.995474 21559 consensus_queue.cc:237] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } }
I20260809 01:03:18.995782 21555 sys_catalog.cc:564] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:03:18.998049 21562 sys_catalog.cc:455] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c842fa5705564b6ab185134c3534ffa8. Latest consensus state: current_term: 2 leader_uuid: "c842fa5705564b6ab185134c3534ffa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } } }
I20260809 01:03:18.998505 21562 sys_catalog.cc:458] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: This master's current role is: LEADER
I20260809 01:03:18.999055 21561 sys_catalog.cc:455] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c842fa5705564b6ab185134c3534ffa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } } }
I20260809 01:03:18.999461 21561 sys_catalog.cc:458] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: This master's current role is: LEADER
I20260809 01:03:19.007001 21565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:03:19.013095 21565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:03:19.014087 21565 catalog_manager.cc:1259] Loaded cluster ID: fbb136614e4f4fc9911895cc99c8cd08
I20260809 01:03:19.014204 21565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:03:19.016748 21565 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:03:19.017841 21565 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Loaded TSK: 0
I20260809 01:03:19.040190 21565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Generated new TSK 1
I20260809 01:03:19.040491 21565 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 01:03:19.303800 21557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:03:19.304111 21557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:03:19.304288 21557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:03:19.311390 21557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:03:19.311527 21557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:03:19.311697 21557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:03:19.321197 21557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:41329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=35905
--tserver_master_addrs=127.4.82.190:34755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:03:19.322413 21557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:03:19.323844 21557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:03:19.336622 21585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:19.336683 21586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:19.337621 21588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:03:19.337702 21557 server_base.cc:1034] running on GCE node
I20260809 01:03:19.338335 21557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:03:19.339320 21557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:03:19.340538 21557 hybrid_clock.cc:648] HybridClock initialized: now 1786237399340499 us; error 37 us; skew 500 ppm
I20260809 01:03:19.340844 21557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:03:19 dist-test-slave-07c2 krb5kdc[20910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237399, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:03:19.343968 21557 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:03:19.345404 21557 webserver.cc:460] Webserver started at http://127.4.82.129:35905/ using document root <none> and password file <none>
I20260809 01:03:19.345973 21557 fs_manager.cc:359] Metadata directory not provided
I20260809 01:03:19.346084 21557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:03:19.351419 21557 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 01:03:19.354214 21596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:19.355530 21557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:03:19.355702 21557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "6c57020af77c48fdadb7155d662472d7"
format_stamp: "Formatted at 2026-08-09 01:02:56 on dist-test-slave-07c2"
server_key: "3d367208faf39b8e19cbe9098b650ccb"
server_key_iv: "73ce5ae85a9ca51bf9c0e0c6d739fc3b"
server_key_version: "encryptionkey@0"
I20260809 01:03:19.356593 21557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:03:19.372264 21557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:03:19.376386 21557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:03:19.376667 21557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:03:19.377547 21557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:03:19.379179 21603 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260809 01:03:19.384195 21603 ts_tablet_manager.cc:1852] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260809 01:03:19.385090 21603 ts_tablet_manager.cc:1905] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:03:19.391984 21603 ts_tablet_manager.cc:1918] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:03:19.392180 21603 log.cc:1196] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/75cd3b0a6b82436580247bbf15afade2
I20260809 01:03:19.392678 21603 ts_tablet_manager.cc:1939] T 75cd3b0a6b82436580247bbf15afade2 P 6c57020af77c48fdadb7155d662472d7: Deleting consensus metadata
I20260809 01:03:19.393038 21557 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260809 01:03:19.393169 21557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.014s user 0.001s sys 0.000s
I20260809 01:03:19.393307 21557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:03:19.393395 21557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:19.426029 21557 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:41329
I20260809 01:03:19.426107 21722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:41329 every 8 connection(s)
I20260809 01:03:19.427258 21557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:03:19.434935 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 21557
I20260809 01:03:19.435775 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:44101
--local_ip_for_outbound_sockets=127.4.82.130
--tserver_master_addrs=127.4.82.190:34755
--webserver_port=38217
--webserver_interface=127.4.82.130
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 01:03:19 dist-test-slave-07c2 krb5kdc[20910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237399, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:03:19.451597 21726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:19.430019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:45837 (local address 127.4.82.190:34755)
0809 01:03:19.430674 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:19.430690 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:19.431055 (+ 365us) server_negotiation.cc:407] Connection header received
0809 01:03:19.433105 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:19.433117 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:19.433417 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:19.433681 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:03:19.435079 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:19.437051 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:19.438555 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:19.439683 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:19.439874 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:19.443711 (+ 3837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:03:19.443722 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:03:19.443739 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:03:19.443789 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:03:19.447393 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:19.448278 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:19.448284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:19.448289 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:19.448442 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:19.448928 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:19.448931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:19.448933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:19.449359 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:03:19.449497 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:19.450368 (+ 871us) server_negotiation.cc:299] Negotiation successful
0809 01:03:19.450671 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":261,"threads_started":1}
I20260809 01:03:19.452090 21727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:19.430139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:45837)
0809 01:03:19.430889 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:19.430924 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 01:03:19.432611 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:03:19.433821 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:03:19.433828 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:03:19.434359 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:03:19.434876 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:19.434892 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:19.437212 (+ 2320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:19.437216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:19.438406 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:19.438418 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:19.439879 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:19.439883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:19.440061 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:19.440744 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:03:19.440765 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:03:19.443450 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:03:19.447534 (+ 4084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:03:19.447545 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:03:19.447557 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:03:19.448139 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:03:19.448578 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:03:19.448586 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:03:19.448593 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:03:19.448821 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:03:19.449513 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:03:19.449524 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:03:19.449917 (+ 393us) client_negotiation.cc:769] Sending connection context
0809 01:03:19.450223 (+ 306us) client_negotiation.cc:240] Negotiation successful
0809 01:03:19.450799 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":313,"threads_started":1}
I20260809 01:03:19.454916 21724 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34755
I20260809 01:03:19.455227 21724 heartbeater.cc:464] Registering TS with master...
I20260809 01:03:19.456063 21724 heartbeater.cc:511] Master 127.4.82.190:34755 requested a full tablet report, sending...
I20260809 01:03:19.458455 21518 ts_manager.cc:194] Registered new tserver with Master: 6c57020af77c48fdadb7155d662472d7 (127.4.82.129:41329)
I20260809 01:03:19.460705 21518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:45837
W20260809 01:03:19.764277 21728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:03:19.764514 21728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:03:19.764608 21728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:03:19.771248 21728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:03:19.771348 21728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:03:19.771502 21728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:03:19.780287 21728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:44101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=38217
--tserver_master_addrs=127.4.82.190:34755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:03:19.781288 21728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:03:19.782437 21728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:03:19.794806 21733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:19.794726 21736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:19.794751 21734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:03:19.795588 21728 server_base.cc:1034] running on GCE node
I20260809 01:03:19.796276 21728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:03:19.797149 21728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:03:19.798369 21728 hybrid_clock.cc:648] HybridClock initialized: now 1786237399798313 us; error 48 us; skew 500 ppm
I20260809 01:03:19.798663 21728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:03:19 dist-test-slave-07c2 krb5kdc[20910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237399, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:03:19.801702 21728 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:03:19.803007 21728 webserver.cc:460] Webserver started at http://127.4.82.130:38217/ using document root <none> and password file <none>
I20260809 01:03:19.803725 21728 fs_manager.cc:359] Metadata directory not provided
I20260809 01:03:19.803838 21728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:03:19.808813 21728 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 01:03:19.811147 21744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:19.812347 21728 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 01:03:19.812510 21728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "78f474c1c85f4ab089294cf4ea52db06"
format_stamp: "Formatted at 2026-08-09 01:02:57 on dist-test-slave-07c2"
server_key: "7ca9290d15e281714d577512e272ff65"
server_key_iv: "113fd0c356f20bb9dd771b5c76127b06"
server_key_version: "encryptionkey@0"
I20260809 01:03:19.813382 21728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:03:19.843770 21728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:03:19.847875 21728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:03:19.848145 21728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:03:19.849076 21728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:03:19.850309 21728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:03:19.850417 21728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:19.850543 21728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:03:19.850638 21728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:19.883368 21728 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:44101
I20260809 01:03:19.883390 21862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:44101 every 8 connection(s)
I20260809 01:03:19.884819 21728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:03:19.887409 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 21728
I20260809 01:03:19.888193 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:39695
--local_ip_for_outbound_sockets=127.4.82.131
--tserver_master_addrs=127.4.82.190:34755
--webserver_port=36825
--webserver_interface=127.4.82.131
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 01:03:19 dist-test-slave-07c2 krb5kdc[20910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237399, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:03:19.907817 21726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:19.887388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:56265 (local address 127.4.82.190:34755)
0809 01:03:19.887589 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:19.887594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:19.889130 (+ 1536us) server_negotiation.cc:407] Connection header received
0809 01:03:19.891170 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:19.891175 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:19.891273 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:19.891426 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:03:19.893472 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:19.894124 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:19.896642 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:19.897087 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:19.897181 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:19.901745 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:03:19.901750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:03:19.901753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:03:19.901785 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:03:19.904531 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:19.905458 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:19.905462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:19.905464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:19.905581 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:19.906122 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:19.906126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:19.906128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:19.906400 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:03:19.906527 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:19.907396 (+ 869us) server_negotiation.cc:299] Negotiation successful
0809 01:03:19.907570 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 01:03:19.909253 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:19.887983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:56265)
0809 01:03:19.888901 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:19.888945 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 01:03:19.890893 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:03:19.891686 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:03:19.891698 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:03:19.892471 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:03:19.893278 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:19.893316 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:19.894286 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:19.894295 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:19.896482 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:19.896501 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:19.897321 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:19.897330 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:19.897645 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:19.898353 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:03:19.898387 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:03:19.901481 (+ 3094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:03:19.904686 (+ 3205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:03:19.904697 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:03:19.904713 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:03:19.905335 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:03:19.905697 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:03:19.905705 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:03:19.905710 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:03:19.905975 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:03:19.906532 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:03:19.906542 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:03:19.906961 (+ 419us) client_negotiation.cc:769] Sending connection context
0809 01:03:19.907308 (+ 347us) client_negotiation.cc:240] Negotiation successful
0809 01:03:19.907851 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":381,"threads_started":1}
I20260809 01:03:19.910646 21863 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34755
I20260809 01:03:19.910959 21863 heartbeater.cc:464] Registering TS with master...
I20260809 01:03:19.911681 21863 heartbeater.cc:511] Master 127.4.82.190:34755 requested a full tablet report, sending...
I20260809 01:03:19.913107 21518 ts_manager.cc:194] Registered new tserver with Master: 78f474c1c85f4ab089294cf4ea52db06 (127.4.82.130:44101)
I20260809 01:03:19.913908 21518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:56265
W20260809 01:03:20.234543 21866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:03:20.234794 21866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:03:20.234886 21866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:03:20.242374 21866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:03:20.242471 21866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:03:20.242588 21866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:03:20.251814 21866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:39695
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=36825
--tserver_master_addrs=127.4.82.190:34755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:03:20.252894 21866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:03:20.254036 21866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:03:20.266685 21871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:20.266767 21874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:20.266770 21872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:03:20.267551 21866 server_base.cc:1034] running on GCE node
I20260809 01:03:20.268122 21866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:03:20.268932 21866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:03:20.270131 21866 hybrid_clock.cc:648] HybridClock initialized: now 1786237400270084 us; error 29 us; skew 500 ppm
I20260809 01:03:20.270434 21866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:03:20 dist-test-slave-07c2 krb5kdc[20910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237400, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:03:20.273433 21866 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:03:20.274710 21866 webserver.cc:460] Webserver started at http://127.4.82.131:36825/ using document root <none> and password file <none>
I20260809 01:03:20.275382 21866 fs_manager.cc:359] Metadata directory not provided
I20260809 01:03:20.275533 21866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:03:20.280465 21866 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 01:03:20.282925 21882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:20.284106 21866 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 01:03:20.284279 21866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "16279b6f436543a8905c001ad14f4309"
format_stamp: "Formatted at 2026-08-09 01:02:57 on dist-test-slave-07c2"
server_key: "884e9ae10c1aa8c356830ccec0583af7"
server_key_iv: "f8977ec604eeaacc242a7966fe8cd219"
server_key_version: "encryptionkey@0"
I20260809 01:03:20.285202 21866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:03:20.308005 21866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:03:20.311774 21866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:03:20.312026 21866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:03:20.312923 21866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:03:20.314184 21866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:03:20.314345 21866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:20.314461 21866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:03:20.314548 21866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:20.347879 21866 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:39695
I20260809 01:03:20.347960 21996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:39695 every 8 connection(s)
I20260809 01:03:20.349239 21866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:03:20.359457 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 21866
Aug 09 01:03:20 dist-test-slave-07c2 krb5kdc[20910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237400, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:03:20.368702 21726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.351744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:36107 (local address 127.4.82.190:34755)
0809 01:03:20.351931 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:20.351935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:20.352976 (+ 1041us) server_negotiation.cc:407] Connection header received
0809 01:03:20.354722 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:20.354728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:20.354818 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:20.354925 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:03:20.356372 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.357063 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.358568 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.359048 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.359194 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.362995 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:03:20.362999 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:03:20.363003 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:03:20.363032 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:03:20.365790 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:20.366479 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:20.366486 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:20.366489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:20.366629 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:20.367051 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:20.367057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:20.367060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:20.367370 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:03:20.367516 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:20.368131 (+ 615us) server_negotiation.cc:299] Negotiation successful
0809 01:03:20.368342 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 01:03:20.369565 22000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.352114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:36107)
0809 01:03:20.352807 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:20.352849 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 01:03:20.354464 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:03:20.355149 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:03:20.355157 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:03:20.355713 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:03:20.356205 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:20.356221 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.357221 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.357232 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:20.358421 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:20.358433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.359397 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.359403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:20.359644 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.360374 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:03:20.360407 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:03:20.362781 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:03:20.365942 (+ 3161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:03:20.365950 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:03:20.365962 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:03:20.366362 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:03:20.366755 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:03:20.366761 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:03:20.366765 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:03:20.366939 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:03:20.367488 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:03:20.367495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:03:20.367800 (+ 305us) client_negotiation.cc:769] Sending connection context
0809 01:03:20.368049 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 01:03:20.368512 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":266,"threads_started":1}
I20260809 01:03:20.371011 21997 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34755
I20260809 01:03:20.371325 21997 heartbeater.cc:464] Registering TS with master...
I20260809 01:03:20.372056 21997 heartbeater.cc:511] Master 127.4.82.190:34755 requested a full tablet report, sending...
I20260809 01:03:20.373456 21518 ts_manager.cc:194] Registered new tserver with Master: 16279b6f436543a8905c001ad14f4309 (127.4.82.131:39695)
I20260809 01:03:20.374245 21518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:36107
I20260809 01:03:20.375010 22001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.360390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40274 (local address 127.4.82.190:34755)
0809 01:03:20.360986 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:20.360994 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:20.361353 (+ 359us) server_negotiation.cc:407] Connection header received
0809 01:03:20.361539 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:20.361543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:20.361616 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:20.361725 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:03:20.363397 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.364334 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.365729 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.366159 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.366290 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.369102 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:03:20.369106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:03:20.369109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:03:20.369136 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:03:20.372311 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:20.372985 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:20.372990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:20.372993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:20.373122 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:20.373498 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:20.373504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:20.373507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:20.373794 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:03:20.374080 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:20.374391 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 01:03:20.374612 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":347,"threads_started":1}
I20260809 01:03:20.377523 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:03:20.383741 22001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.378454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40286 (local address 127.4.82.190:34755)
0809 01:03:20.378648 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:20.378654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:20.378945 (+ 291us) server_negotiation.cc:407] Connection header received
0809 01:03:20.379162 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:20.379165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:20.379272 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:20.379453 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:03:20.379873 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.380549 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.381941 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.382335 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.382456 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.382918 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:03:20.383127 (+ 209us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 01:03:20.383487 (+ 360us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40286: token expired
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 01:03:20.383824 22001 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40286: token expired
W20260809 01:03:20.384369 21435 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.82.190:34755)
I20260809 01:03:20.397102 22001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.384830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40302 (local address 127.4.82.190:34755)
0809 01:03:20.384990 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:20.384998 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:20.385014 (+ 16us) server_negotiation.cc:407] Connection header received
0809 01:03:20.385181 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:20.385187 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:20.385247 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:20.385341 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:03:20.386646 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.387406 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.388894 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.389367 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.389489 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.392318 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:03:20.392324 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:03:20.392329 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:03:20.392368 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:03:20.394535 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:20.395165 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:20.395171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:20.395174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:20.395384 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:20.395772 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:20.395777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:20.395781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:20.396049 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:03:20.396293 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:20.396587 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 01:03:20.396788 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 01:03:20.399996 21518 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:03:20.407061 21518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:03:20.438539 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.419142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45600 (local address 127.4.82.130:44101)
0809 01:03:20.430395 (+ 11253us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:20.430406 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:20.430442 (+ 36us) server_negotiation.cc:407] Connection header received
0809 01:03:20.430554 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:20.430565 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:20.430864 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:20.431091 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:03:20.431740 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.433326 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.436293 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.437252 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.437414 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.437609 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:20.437937 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 01:03:20.438144 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11095,"thread_start_us":10989,"threads_started":1}
I20260809 01:03:20.439337 22013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.418229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:44101 (local address 127.0.0.1:45600)
0809 01:03:20.420508 (+ 2279us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:20.420528 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:03:20.420658 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:03:20.431109 (+ 10451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:03:20.431116 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:03:20.431159 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:03:20.431521 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:20.431531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.433555 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.433561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:20.436100 (+ 2539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:20.436118 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.437483 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.437490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:20.437703 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.437745 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 01:03:20.437870 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 01:03:20.438231 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2130,"thread_start_us":888,"threads_started":1}
I20260809 01:03:20.441954 22008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.419939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:39806)
0809 01:03:20.430260 (+ 10321us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:20.430277 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:03:20.430381 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:03:20.433678 (+ 3297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:03:20.433686 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:03:20.433725 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:03:20.434272 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:20.434293 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.436135 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.436140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:20.439255 (+ 3115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:20.439270 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.441130 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.441146 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:20.441321 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.441353 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 01:03:20.441467 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 01:03:20.441584 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10213,"mutex_wait_us":46,"thread_start_us":702,"threads_started":1}
I20260809 01:03:20.441996 22014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.420467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39806 (local address 127.4.82.129:41329)
0809 01:03:20.432764 (+ 12297us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:20.432775 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:20.432808 (+ 33us) server_negotiation.cc:407] Connection header received
0809 01:03:20.432952 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:20.432962 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:20.433209 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:20.434509 (+ 1300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:03:20.434741 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.435982 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.439472 (+ 3490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.440923 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.441069 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.441286 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:20.441507 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 01:03:20.441665 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12092,"thread_start_us":12927,"threads_started":1}
I20260809 01:03:20.442934 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.417915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:39695 (local address 127.0.0.1:39596)
0809 01:03:20.419618 (+ 1703us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:20.419682 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 01:03:20.419861 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:03:20.435731 (+ 15870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:03:20.435736 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:03:20.435760 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:03:20.436258 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:20.436274 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.438213 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.438218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:20.440411 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:20.440424 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.442187 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.442193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:20.442372 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.442406 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 01:03:20.442485 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 01:03:20.442598 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1396,"mutex_wait_us":3,"thread_start_us":12510,"threads_started":1}
I20260809 01:03:20.443966 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.418541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39596 (local address 127.4.82.131:39695)
0809 01:03:20.435178 (+ 16637us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:20.435185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:20.435214 (+ 29us) server_negotiation.cc:407] Connection header received
0809 01:03:20.435324 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:20.435331 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:20.435518 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:20.435715 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:03:20.436460 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.438026 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.440573 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.441950 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.443151 (+ 1201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.443400 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:20.443463 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 01:03:20.443618 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16483,"thread_start_us":16324,"threads_started":1}
I20260809 01:03:20.443608 21794 tablet_service.cc:1467] Processing CreateTablet for tablet 6104aae2bd8c4283be7d53574d9add64 (DEFAULT_TABLE table=test-table [id=55d8ad525d34446aac60382fdc16bcaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:03:20.444934 21794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6104aae2bd8c4283be7d53574d9add64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:03:20.448302 21650 tablet_service.cc:1467] Processing CreateTablet for tablet 6104aae2bd8c4283be7d53574d9add64 (DEFAULT_TABLE table=test-table [id=55d8ad525d34446aac60382fdc16bcaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:03:20.449541 21650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6104aae2bd8c4283be7d53574d9add64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:03:20.450449 21930 tablet_service.cc:1467] Processing CreateTablet for tablet 6104aae2bd8c4283be7d53574d9add64 (DEFAULT_TABLE table=test-table [id=55d8ad525d34446aac60382fdc16bcaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:03:20.451495 21930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6104aae2bd8c4283be7d53574d9add64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:03:20.459327 22015 tablet_bootstrap.cc:492] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7: Bootstrap starting.
I20260809 01:03:20.461292 22016 tablet_bootstrap.cc:492] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06: Bootstrap starting.
I20260809 01:03:20.463570 22017 tablet_bootstrap.cc:492] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309: Bootstrap starting.
I20260809 01:03:20.464659 22015 tablet_bootstrap.cc:654] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7: Neither blocks nor log segments found. Creating new log.
I20260809 01:03:20.464708 21724 heartbeater.cc:503] Master 127.4.82.190:34755 was elected leader, sending a full tablet report...
I20260809 01:03:20.464769 22016 tablet_bootstrap.cc:654] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06: Neither blocks nor log segments found. Creating new log.
I20260809 01:03:20.466014 22015 log.cc:824] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7: Log is configured to *not* fsync() on all Append() calls
I20260809 01:03:20.466009 22016 log.cc:824] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06: Log is configured to *not* fsync() on all Append() calls
I20260809 01:03:20.467056 22017 tablet_bootstrap.cc:654] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309: Neither blocks nor log segments found. Creating new log.
I20260809 01:03:20.468380 22017 log.cc:824] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309: Log is configured to *not* fsync() on all Append() calls
I20260809 01:03:20.471680 22017 tablet_bootstrap.cc:492] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309: No bootstrap required, opened a new log
I20260809 01:03:20.471907 22016 tablet_bootstrap.cc:492] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06: No bootstrap required, opened a new log
I20260809 01:03:20.471953 22017 ts_tablet_manager.cc:1397] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260809 01:03:20.472074 22015 tablet_bootstrap.cc:492] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7: No bootstrap required, opened a new log
I20260809 01:03:20.472154 22016 ts_tablet_manager.cc:1397] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260809 01:03:20.472332 22015 ts_tablet_manager.cc:1397] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.006s
I20260809 01:03:20.478902 22015 raft_consensus.cc:348] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } }
I20260809 01:03:20.479240 22015 raft_consensus.cc:374] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:03:20.479383 22015 raft_consensus.cc:729] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c57020af77c48fdadb7155d662472d7, State: Initialized, Role: FOLLOWER
I20260809 01:03:20.479970 22015 consensus_queue.cc:260] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } }
I20260809 01:03:20.482540 22016 raft_consensus.cc:348] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } }
I20260809 01:03:20.482963 22016 raft_consensus.cc:374] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:03:20.483023 22015 ts_tablet_manager.cc:1428] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260809 01:03:20.483057 22016 raft_consensus.cc:729] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78f474c1c85f4ab089294cf4ea52db06, State: Initialized, Role: FOLLOWER
I20260809 01:03:20.482824 22017 raft_consensus.cc:348] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } }
I20260809 01:03:20.483222 22017 raft_consensus.cc:374] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:03:20.483323 22017 raft_consensus.cc:729] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16279b6f436543a8905c001ad14f4309, State: Initialized, Role: FOLLOWER
I20260809 01:03:20.483698 22016 consensus_queue.cc:260] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } }
I20260809 01:03:20.483958 22017 consensus_queue.cc:260] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } }
I20260809 01:03:20.484848 21863 heartbeater.cc:503] Master 127.4.82.190:34755 was elected leader, sending a full tablet report...
I20260809 01:03:20.487041 22016 ts_tablet_manager.cc:1428] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06: Time spent starting tablet: real 0.015s user 0.010s sys 0.003s
I20260809 01:03:20.490909 21997 heartbeater.cc:503] Master 127.4.82.190:34755 was elected leader, sending a full tablet report...
I20260809 01:03:20.491601 22017 ts_tablet_manager.cc:1428] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309: Time spent starting tablet: real 0.020s user 0.014s sys 0.005s
W20260809 01:03:20.600778 21999 tablet.cc:2369] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 01:03:20.636855 21864 tablet.cc:2369] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:03:20.676676 22024 raft_consensus.cc:482] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:03:20.676949 22024 raft_consensus.cc:504] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } }
I20260809 01:03:20.678570 22024 leader_election.cc:288] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78f474c1c85f4ab089294cf4ea52db06 (127.4.82.130:44101), 6c57020af77c48fdadb7155d662472d7 (127.4.82.129:41329)
W20260809 01:03:20.679644 21725 tablet.cc:2369] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:03:20.684566 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.679409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:33379 (local address 127.4.82.130:44101)
0809 01:03:20.679630 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:20.679637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:20.679658 (+ 21us) server_negotiation.cc:407] Connection header received
0809 01:03:20.679740 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:20.679748 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:20.679830 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:20.679922 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:03:20.680480 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.681202 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.682850 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.683690 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.683858 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.683930 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:20.684194 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 01:03:20.684321 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 01:03:20.684595 22000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.678797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:44101 (local address 127.4.82.131:33379)
0809 01:03:20.679006 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:20.679024 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 01:03:20.679132 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:03:20.680107 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:03:20.680111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:03:20.680137 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:03:20.680351 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:20.680364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.681356 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.681364 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:20.682670 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:20.682682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.683837 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.683843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:20.684034 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.684058 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 01:03:20.684167 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 01:03:20.684296 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":2}
I20260809 01:03:20.685482 21815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6104aae2bd8c4283be7d53574d9add64" candidate_uuid: "16279b6f436543a8905c001ad14f4309" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78f474c1c85f4ab089294cf4ea52db06" is_pre_election: true
I20260809 01:03:20.685647 22014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.679409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:43793 (local address 127.4.82.129:41329)
0809 01:03:20.679632 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:20.679637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:20.680374 (+ 737us) server_negotiation.cc:407] Connection header received
0809 01:03:20.680590 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:20.680594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:20.680652 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:20.680753 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:03:20.681272 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.681956 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.683664 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.684425 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.684622 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.684717 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:20.685273 (+ 556us) server_negotiation.cc:299] Negotiation successful
0809 01:03:20.685401 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 01:03:20.685745 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.679222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.4.82.131:43793)
0809 01:03:20.680241 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:20.680262 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:03:20.680378 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:03:20.680767 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:03:20.680771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:03:20.680794 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:03:20.681109 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:20.681117 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.682103 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.682109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:20.683513 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:03:20.683525 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.684910 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.684922 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:03:20.685117 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.685142 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 01:03:20.685246 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 01:03:20.685365 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"mutex_wait_us":1,"thread_start_us":680,"threads_started":1}
I20260809 01:03:20.685986 21815 raft_consensus.cc:2454] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16279b6f436543a8905c001ad14f4309 in term 0.
I20260809 01:03:20.686602 21883 leader_election.cc:302] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16279b6f436543a8905c001ad14f4309, 78f474c1c85f4ab089294cf4ea52db06; no voters:
I20260809 01:03:20.686772 21672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6104aae2bd8c4283be7d53574d9add64" candidate_uuid: "16279b6f436543a8905c001ad14f4309" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c57020af77c48fdadb7155d662472d7" is_pre_election: true
I20260809 01:03:20.686972 22024 raft_consensus.cc:2781] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:03:20.687130 22024 raft_consensus.cc:482] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:03:20.687243 22024 raft_consensus.cc:3037] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:03:20.687379 21672 raft_consensus.cc:2454] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16279b6f436543a8905c001ad14f4309 in term 0.
I20260809 01:03:20.690179 22024 raft_consensus.cc:504] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } }
I20260809 01:03:20.690850 22024 leader_election.cc:288] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [CANDIDATE]: Term 1 election: Requested vote from peers 78f474c1c85f4ab089294cf4ea52db06 (127.4.82.130:44101), 6c57020af77c48fdadb7155d662472d7 (127.4.82.129:41329)
I20260809 01:03:20.691202 21815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6104aae2bd8c4283be7d53574d9add64" candidate_uuid: "16279b6f436543a8905c001ad14f4309" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78f474c1c85f4ab089294cf4ea52db06"
I20260809 01:03:20.691359 21672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6104aae2bd8c4283be7d53574d9add64" candidate_uuid: "16279b6f436543a8905c001ad14f4309" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c57020af77c48fdadb7155d662472d7"
I20260809 01:03:20.691478 21815 raft_consensus.cc:3037] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:03:20.691597 21672 raft_consensus.cc:3037] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:03:20.693750 21815 raft_consensus.cc:2454] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16279b6f436543a8905c001ad14f4309 in term 1.
I20260809 01:03:20.693804 21672 raft_consensus.cc:2454] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16279b6f436543a8905c001ad14f4309 in term 1.
I20260809 01:03:20.694146 21883 leader_election.cc:302] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16279b6f436543a8905c001ad14f4309, 78f474c1c85f4ab089294cf4ea52db06; no voters:
I20260809 01:03:20.694449 22024 raft_consensus.cc:2781] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:03:20.694900 22024 raft_consensus.cc:686] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [term 1 LEADER]: Becoming Leader. State: Replica: 16279b6f436543a8905c001ad14f4309, State: Running, Role: LEADER
I20260809 01:03:20.695405 22024 consensus_queue.cc:237] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } }
I20260809 01:03:20.700150 21517 catalog_manager.cc:5495] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16279b6f436543a8905c001ad14f4309 (127.4.82.131). New cstate: current_term: 1 leader_uuid: "16279b6f436543a8905c001ad14f4309" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16279b6f436543a8905c001ad14f4309" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 39695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 } health_report { overall_health: UNKNOWN } } }
I20260809 01:03:20.734544 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.729242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39602 (local address 127.4.82.131:39695)
0809 01:03:20.729443 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:20.729450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:20.729470 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:03:20.729728 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:20.729732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:20.729807 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:20.729952 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:03:20.730329 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.731041 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.732517 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.732912 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.733045 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.733556 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:03:20.734007 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:03:20.734152 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:20.734236 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 01:03:20.734308 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 01:03:20.741636 21672 raft_consensus.cc:1261] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7 [term 1 FOLLOWER]: Refusing update from remote peer 16279b6f436543a8905c001ad14f4309: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:03:20.741660 21815 raft_consensus.cc:1261] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06 [term 1 FOLLOWER]: Refusing update from remote peer 16279b6f436543a8905c001ad14f4309: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:03:20.742508 22026 consensus_queue.cc:1035] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c57020af77c48fdadb7155d662472d7" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 41329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:03:20.742887 22024 consensus_queue.cc:1035] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78f474c1c85f4ab089294cf4ea52db06" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 44101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:03:20.760718 22031 mvcc.cc:205] Tried to move back new op lower bound from 7316428393429303296 to 7316428393252474880. Current Snapshot: MvccSnapshot[applied={T|T < 7316428393429303296}]
I20260809 01:03:20.764585 22030 mvcc.cc:205] Tried to move back new op lower bound from 7316428393429303296 to 7316428393252474880. Current Snapshot: MvccSnapshot[applied={T|T < 7316428393429303296 or (T in {7316428393429303296})}]
I20260809 01:03:20.771905 21518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:03:20.772198 21518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:03:20.775547 21518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Sending DeleteTablet for 3 replicas of tablet 6104aae2bd8c4283be7d53574d9add64
I20260809 01:03:20.776818 21794 tablet_service.cc:1514] Processing DeleteTablet for tablet 6104aae2bd8c4283be7d53574d9add64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:03:20 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:45600
I20260809 01:03:20.777011 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 21557
I20260809 01:03:20.777266 21650 tablet_service.cc:1514] Processing DeleteTablet for tablet 6104aae2bd8c4283be7d53574d9add64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:03:20 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:39806
I20260809 01:03:20.776700 21930 tablet_service.cc:1514] Processing DeleteTablet for tablet 6104aae2bd8c4283be7d53574d9add64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:03:20 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:39596
I20260809 01:03:20.780737 22040 tablet_replica.cc:330] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7: stopping tablet replica
I20260809 01:03:20.781195 22040 raft_consensus.cc:2229] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:03:20.781744 22040 raft_consensus.cc:2258] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:03:20.789773 22040 ts_tablet_manager.cc:1905] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:03:20.797902 22014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.777961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39814 (local address 127.4.82.129:41329)
0809 01:03:20.778183 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:20.778191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:20.778215 (+ 24us) server_negotiation.cc:407] Connection header received
0809 01:03:20.778284 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:20.778292 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:20.778397 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:20.778543 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:03:20.780407 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.782143 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.784195 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.785139 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.785303 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.788969 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:03:20.788986 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:03:20.789005 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:03:20.789054 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:03:20.794054 (+ 5000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:20.794907 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:20.794919 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:20.794925 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:20.795164 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:20.795614 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:20.795622 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:20.795628 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:20.796216 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:03:20.796780 (+ 564us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:20.796978 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 01:03:20.797262 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 01:03:20.798723 21717 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:03:20.806784 22041 tablet_replica.cc:330] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309: stopping tablet replica
I20260809 01:03:20.807420 22041 raft_consensus.cc:2229] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:03:20.808221 22041 raft_consensus.cc:2258] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:03:20.809232 22039 tablet_replica.cc:330] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06: stopping tablet replica
I20260809 01:03:20.809641 22039 raft_consensus.cc:2229] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:03:20.810142 22039 raft_consensus.cc:2258] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:03:20.811797 22041 ts_tablet_manager.cc:1905] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:03:20.812302 22039 ts_tablet_manager.cc:1905] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:03:20.819667 22041 ts_tablet_manager.cc:1918] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:03:20.819672 22039 ts_tablet_manager.cc:1918] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:03:20.819844 22041 log.cc:1196] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/6104aae2bd8c4283be7d53574d9add64
I20260809 01:03:20.820003 22039 log.cc:1196] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/6104aae2bd8c4283be7d53574d9add64
I20260809 01:03:20.820502 22041 ts_tablet_manager.cc:1939] T 6104aae2bd8c4283be7d53574d9add64 P 16279b6f436543a8905c001ad14f4309: Deleting consensus metadata
I20260809 01:03:20.820778 22039 ts_tablet_manager.cc:1939] T 6104aae2bd8c4283be7d53574d9add64 P 78f474c1c85f4ab089294cf4ea52db06: Deleting consensus metadata
I20260809 01:03:20.822328 21505 catalog_manager.cc:4841] TS 16279b6f436543a8905c001ad14f4309 (127.4.82.131:39695): tablet 6104aae2bd8c4283be7d53574d9add64 (table test-table [id=55d8ad525d34446aac60382fdc16bcaf]) successfully deleted
I20260809 01:03:20.822621 21502 catalog_manager.cc:4841] TS 78f474c1c85f4ab089294cf4ea52db06 (127.4.82.130:44101): tablet 6104aae2bd8c4283be7d53574d9add64 (table test-table [id=55d8ad525d34446aac60382fdc16bcaf]) successfully deleted
I20260809 01:03:20.903766 22040 ts_tablet_manager.cc:1918] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:03:20.903985 22040 log.cc:1196] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/6104aae2bd8c4283be7d53574d9add64
I20260809 01:03:20.904534 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 21557
I20260809 01:03:20.904584 22040 ts_tablet_manager.cc:1939] T 6104aae2bd8c4283be7d53574d9add64 P 6c57020af77c48fdadb7155d662472d7: Deleting consensus metadata
W20260809 01:03:20.925107 21502 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.82.129:41329 (error 108)
I20260809 01:03:20.926381 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 21728
I20260809 01:03:20.926849 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.926370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:39824)
0809 01:03:20.926567 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:20.926680 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 01:03:20.927112 21502 catalog_manager.cc:4584] TS 6c57020af77c48fdadb7155d662472d7 (127.4.82.129:41329): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6104aae2bd8c4283be7d53574d9add64: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
I20260809 01:03:20.947108 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.927449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45608 (local address 127.4.82.130:44101)
0809 01:03:20.927652 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:20.927658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:20.927693 (+ 35us) server_negotiation.cc:407] Connection header received
0809 01:03:20.927758 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:20.927762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:20.927818 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:20.927925 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:03:20.929366 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.930141 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.931751 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:20.932155 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:20.932290 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:20.939737 (+ 7447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:03:20.939755 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:03:20.939774 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:03:20.939822 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:03:20.943812 (+ 3990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:20.944672 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:20.944684 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:20.944688 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:20.944831 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:20.945251 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:20.945256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:20.945260 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:20.945750 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:03:20.946032 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:20.946349 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 01:03:20.946566 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 01:03:20.947882 21858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:03:20.980496 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.980090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:39838)
0809 01:03:20.980274 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:20.980374 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 01:03:20.981527 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:20.981170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:39850)
0809 01:03:20.981331 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:20.981420 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 01:03:21.042482 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:21.041912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:39852)
0809 01:03:21.042170 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:21.042283 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 01:03:21.043834 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:21.043388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:39856)
0809 01:03:21.043582 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:21.043685 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 01:03:21.054706 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 21728
I20260809 01:03:21.073338 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 21866
I20260809 01:03:21.088438 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:21.074309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39616 (local address 127.4.82.131:39695)
0809 01:03:21.074509 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:21.074515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:21.074535 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:03:21.074646 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:21.074651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:21.074734 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:21.074852 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:03:21.076366 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:21.077201 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:21.078590 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:21.079072 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:21.079222 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:21.081959 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:03:21.081970 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:03:21.081985 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:03:21.082024 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:03:21.085173 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:21.085879 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:21.085897 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:21.085899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:21.086047 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:21.086431 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:21.086437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:21.086440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:21.086796 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:03:21.087144 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:21.087589 (+ 445us) server_negotiation.cc:299] Negotiation successful
0809 01:03:21.087850 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 01:03:21.089567 21992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:03:21.124868 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:21.124296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:39868)
0809 01:03:21.124534 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:21.124650 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 01:03:21.126010 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:21.125552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:39874)
0809 01:03:21.125796 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:21.125883 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260809 01:03:21.211791 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 21866
I20260809 01:03:21.233242 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 21485
I20260809 01:03:21.234397 21550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:03:21.345309 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 21485
I20260809 01:03:21.345433 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:21.344557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:41329 (local address 127.0.0.1:39884)
0809 01:03:21.345093 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:21.345227 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378}
I20260809 01:03:41.369379 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34755
--webserver_interface=127.4.82.190
--webserver_port=46449
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:34755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:03:41.706352 22047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:03:41.706629 22047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:03:41.706823 22047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:03:41.713902 22047 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 01:03:41.713986 22047 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 01:03:41.714080 22047 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 01:03:41.714170 22047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:03:41.714255 22047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:03:41.714365 22047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:03:41.714445 22047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:03:41.714521 22047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:03:41.724138 22047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:34755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=46449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:03:41.725317 22047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:03:41.726562 22047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:03:41.738232 22053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:41.738318 22054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:41.738276 22056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:03:41.739401 22047 server_base.cc:1034] running on GCE node
I20260809 01:03:41.740121 22047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:03:41.741397 22047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:03:41.742646 22047 hybrid_clock.cc:648] HybridClock initialized: now 1786237421742584 us; error 37 us; skew 500 ppm
I20260809 01:03:41.742962 22047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:03:41 dist-test-slave-07c2 krb5kdc[20910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237421, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:03:41.745990 22047 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:03:41.747946 22047 webserver.cc:460] Webserver started at http://127.4.82.190:46449/ using document root <none> and password file <none>
I20260809 01:03:41.748620 22047 fs_manager.cc:359] Metadata directory not provided
I20260809 01:03:41.748745 22047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:03:41.753945 22047 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 01:03:41.756686 22063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:41.757907 22047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 01:03:41.758090 22047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "c842fa5705564b6ab185134c3534ffa8"
format_stamp: "Formatted at 2026-08-09 01:02:55 on dist-test-slave-07c2"
server_key: "ac2a91f20bc804c09d9ab097859b4e9e"
server_key_iv: "6498dd7efd43125383cff29f86024560"
server_key_version: "encryptionkey@0"
I20260809 01:03:41.759013 22047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:03:41.776274 22047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:03:41.779896 22047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:03:41.780179 22047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:03:41.798388 22047 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:34755
I20260809 01:03:41.798424 22117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:34755 every 8 connection(s)
I20260809 01:03:41.799796 22047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:03:41.806387 22118 sys_catalog.cc:263] Verifying existing consensus state
I20260809 01:03:41.807651 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 22047
I20260809 01:03:41.808727 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:41329
--local_ip_for_outbound_sockets=127.4.82.129
--tserver_master_addrs=127.4.82.190:34755
--webserver_port=35905
--webserver_interface=127.4.82.129
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:03:41.811076 22118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Bootstrap starting.
I20260809 01:03:41.833767 22118 log.cc:824] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Log is configured to *not* fsync() on all Append() calls
I20260809 01:03:41.849618 22118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 01:03:41.850140 22118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Bootstrap complete.
I20260809 01:03:41.859141 22118 raft_consensus.cc:348] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } }
I20260809 01:03:41.860081 22118 raft_consensus.cc:729] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c842fa5705564b6ab185134c3534ffa8, State: Initialized, Role: FOLLOWER
I20260809 01:03:41.860740 22118 consensus_queue.cc:260] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } }
I20260809 01:03:41.860968 22118 raft_consensus.cc:388] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:03:41.861099 22118 raft_consensus.cc:482] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:03:41.861276 22118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 2 FOLLOWER]: Advancing to term 3
I20260809 01:03:41.865399 22118 raft_consensus.cc:504] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } }
I20260809 01:03:41.865859 22118 leader_election.cc:302] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c842fa5705564b6ab185134c3534ffa8; no voters:
I20260809 01:03:41.866509 22118 leader_election.cc:288] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 01:03:41.866644 22122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 01:03:41.867955 22122 raft_consensus.cc:686] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [term 3 LEADER]: Becoming Leader. State: Replica: c842fa5705564b6ab185134c3534ffa8, State: Running, Role: LEADER
I20260809 01:03:41.868551 22122 consensus_queue.cc:237] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } }
I20260809 01:03:41.868744 22118 sys_catalog.cc:564] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:03:41.871143 22124 sys_catalog.cc:455] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c842fa5705564b6ab185134c3534ffa8. Latest consensus state: current_term: 3 leader_uuid: "c842fa5705564b6ab185134c3534ffa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } } }
I20260809 01:03:41.871593 22124 sys_catalog.cc:458] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: This master's current role is: LEADER
I20260809 01:03:41.872315 22123 sys_catalog.cc:455] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c842fa5705564b6ab185134c3534ffa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c842fa5705564b6ab185134c3534ffa8" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34755 } } }
I20260809 01:03:41.872650 22123 sys_catalog.cc:458] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8 [sys.catalog]: This master's current role is: LEADER
I20260809 01:03:41.879865 22128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:03:41.887988 22128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:03:41.889460 22128 catalog_manager.cc:1259] Loaded cluster ID: fbb136614e4f4fc9911895cc99c8cd08
I20260809 01:03:41.889585 22128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:03:41.893705 22128 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:03:41.895313 22128 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Loaded TSK: 0
I20260809 01:03:41.895485 22128 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c842fa5705564b6ab185134c3534ffa8: Loaded TSK: 1
F20260809 01:03:41.896576 22128 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fea2d65c9ad google::LogMessage::Fail() at ??:0
@ 0x7fea2d65f88a google::LogMessage::SendToLog() at ??:0
@ 0x7fea2d65c4de google::LogMessage::Flush() at ??:0
@ 0x7fea2d65da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fea313c07d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fea313af73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fea4308b1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fea4308a783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fea431162a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fea431160c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fea431605e9 std::function<>::operator()() at ??:0
@ 0x7fea4308ecdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fea4308cc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fea43114610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fea43114439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fea302a8345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fea302ab91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fea302ab799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fea30286eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fea335606db start_thread at ??:0
@ 0x7fea2c4f661f clone at ??:0
W20260809 01:03:42.189033 22120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:03:42.189296 22120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:03:42.189455 22120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:03:42.196571 22120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:03:42.196710 22120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:03:42.196908 22120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:03:42.206288 22120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:41329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=35905
--tserver_master_addrs=127.4.82.190:34755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:03:42.207511 22120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:03:42.208766 22120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:03:42.221683 22167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:42.221758 22170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:42.221766 22168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:03:42.222582 22120 server_base.cc:1034] running on GCE node
I20260809 01:03:42.223169 22120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:03:42.224056 22120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:03:42.225268 22120 hybrid_clock.cc:648] HybridClock initialized: now 1786237422225226 us; error 36 us; skew 500 ppm
I20260809 01:03:42.225566 22120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:03:42 dist-test-slave-07c2 krb5kdc[20910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237422, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:03:42.228749 22120 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:03:42.230131 22120 webserver.cc:460] Webserver started at http://127.4.82.129:35905/ using document root <none> and password file <none>
I20260809 01:03:42.230770 22120 fs_manager.cc:359] Metadata directory not provided
I20260809 01:03:42.230916 22120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:03:42.235877 22120 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:03:42.238356 22177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:42.239621 22120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 01:03:42.239799 22120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "6c57020af77c48fdadb7155d662472d7"
format_stamp: "Formatted at 2026-08-09 01:02:56 on dist-test-slave-07c2"
server_key: "3d367208faf39b8e19cbe9098b650ccb"
server_key_iv: "73ce5ae85a9ca51bf9c0e0c6d739fc3b"
server_key_version: "encryptionkey@0"
I20260809 01:03:42.240734 22120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:03:42.260771 22120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:03:42.264405 22120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:03:42.264648 22120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:03:42.265533 22120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:03:42.266765 22120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:03:42.266896 22120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:42.267014 22120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:03:42.267098 22120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:42.301842 22120 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:41329
I20260809 01:03:42.301924 22290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:41329 every 8 connection(s)
I20260809 01:03:42.303222 22120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:03:42.307790 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:42.305972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:49811)
0809 01:03:42.306670 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:42.306897 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":283,"threads_started":1}
W20260809 01:03:42.308373 22291 heartbeater.cc:644] Failed to heartbeat to 127.4.82.190:34755 (0 consecutive failures): Network error: Failed to ping master at 127.4.82.190:34755: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
I20260809 01:03:42.308923 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 22120
I20260809 01:03:42.309212 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:42.308884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:52545)
0809 01:03:42.309048 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:42.309117 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 01:03:42.309639 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:44101
--local_ip_for_outbound_sockets=127.4.82.130
--tserver_master_addrs=127.4.82.190:34755
--webserver_port=38217
--webserver_interface=127.4.82.130
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:03:42.310156 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:42.309703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:51497)
0809 01:03:42.309902 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:42.309999 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 01:03:42.310456 22291 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 01:03:42.637892 22294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:03:42.638159 22294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:03:42.638258 22294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:03:42.645443 22294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:03:42.645536 22294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:03:42.645653 22294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:03:42.654783 22294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:44101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=38217
--tserver_master_addrs=127.4.82.190:34755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:03:42.656013 22294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:03:42.657191 22294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:03:42.670316 22300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:42.670437 22302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:42.670349 22299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:03:42.671113 22294 server_base.cc:1034] running on GCE node
I20260809 01:03:42.671739 22294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:03:42.672621 22294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:03:42.673822 22294 hybrid_clock.cc:648] HybridClock initialized: now 1786237422673783 us; error 30 us; skew 500 ppm
I20260809 01:03:42.674090 22294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:03:42 dist-test-slave-07c2 krb5kdc[20910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237422, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:03:42.677119 22294 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:03:42.678580 22294 webserver.cc:460] Webserver started at http://127.4.82.130:38217/ using document root <none> and password file <none>
I20260809 01:03:42.679273 22294 fs_manager.cc:359] Metadata directory not provided
I20260809 01:03:42.679467 22294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:03:42.684635 22294 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 01:03:42.687160 22312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:42.688410 22294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 01:03:42.688591 22294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "78f474c1c85f4ab089294cf4ea52db06"
format_stamp: "Formatted at 2026-08-09 01:02:57 on dist-test-slave-07c2"
server_key: "7ca9290d15e281714d577512e272ff65"
server_key_iv: "113fd0c356f20bb9dd771b5c76127b06"
server_key_version: "encryptionkey@0"
I20260809 01:03:42.689548 22294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:03:42.716197 22294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:03:42.720052 22294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:03:42.720332 22294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:03:42.721289 22294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:03:42.722568 22294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:03:42.722687 22294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:42.722817 22294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:03:42.722908 22294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:42.756011 22294 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:44101
I20260809 01:03:42.756095 22434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:44101 every 8 connection(s)
I20260809 01:03:42.757443 22294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:03:42.762045 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:42.760297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:44969)
0809 01:03:42.760991 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:42.761223 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":274,"threads_started":1}
W20260809 01:03:42.762660 22435 heartbeater.cc:644] Failed to heartbeat to 127.4.82.190:34755 (0 consecutive failures): Network error: Failed to ping master at 127.4.82.190:34755: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
I20260809 01:03:42.763461 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:42.763107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:50237)
0809 01:03:42.763258 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:42.763359 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 01:03:42.764252 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:42.763937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:53241)
0809 01:03:42.764070 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:42.764145 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 01:03:42.764469 22435 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 01:03:42.766847 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 22294
I20260809 01:03:42.767524 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:39695
--local_ip_for_outbound_sockets=127.4.82.131
--tserver_master_addrs=127.4.82.190:34755
--webserver_port=36825
--webserver_interface=127.4.82.131
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:03:43.094408 22438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:03:43.094705 22438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:03:43.094803 22438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:03:43.101938 22438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:03:43.102058 22438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:03:43.102178 22438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:03:43.111783 22438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:39695
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=36825
--tserver_master_addrs=127.4.82.190:34755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:03:43.113086 22438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:03:43.114404 22438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:03:43.127794 22444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:43.127897 22443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:03:43.128180 22446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:03:43.129092 22438 server_base.cc:1034] running on GCE node
I20260809 01:03:43.129742 22438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:03:43.130697 22438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:03:43.131923 22438 hybrid_clock.cc:648] HybridClock initialized: now 1786237423131866 us; error 46 us; skew 500 ppm
I20260809 01:03:43.132246 22438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:03:43 dist-test-slave-07c2 krb5kdc[20910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237423, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:03:43.135270 22438 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:03:43.136689 22438 webserver.cc:460] Webserver started at http://127.4.82.131:36825/ using document root <none> and password file <none>
I20260809 01:03:43.137354 22438 fs_manager.cc:359] Metadata directory not provided
I20260809 01:03:43.137495 22438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:03:43.142602 22438 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260809 01:03:43.145526 22454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:43.146811 22438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 01:03:43.146996 22438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "16279b6f436543a8905c001ad14f4309"
format_stamp: "Formatted at 2026-08-09 01:02:57 on dist-test-slave-07c2"
server_key: "884e9ae10c1aa8c356830ccec0583af7"
server_key_iv: "f8977ec604eeaacc242a7966fe8cd219"
server_key_version: "encryptionkey@0"
I20260809 01:03:43.148041 22438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:03:43.175659 22438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:03:43.180032 22438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:03:43.180332 22438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:03:43.181358 22438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:03:43.182696 22438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:03:43.182832 22438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:43.182955 22438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:03:43.183046 22438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:03:43.219102 22438 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:39695
I20260809 01:03:43.219226 22574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:39695 every 8 connection(s)
I20260809 01:03:43.220927 22438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:03:43.224385 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 22438
I20260809 01:03:43.225975 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:43.224011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:50121)
0809 01:03:43.224824 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:43.225097 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":337,"threads_started":1}
W20260809 01:03:43.226691 22575 heartbeater.cc:644] Failed to heartbeat to 127.4.82.190:34755 (0 consecutive failures): Network error: Failed to ping master at 127.4.82.190:34755: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
I20260809 01:03:43.227463 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:43.227103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:58723)
0809 01:03:43.227249 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:43.227354 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 01:03:43.228216 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:43.227903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:50233)
0809 01:03:43.228034 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:43.228112 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 01:03:43.228456 22575 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 01:03:43.311692 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:43.310891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:46511)
0809 01:03:43.311387 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:43.311471 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":226,"threads_started":1}
I20260809 01:03:43.765723 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:43.764951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:43517)
0809 01:03:43.765394 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:43.765493 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":218,"threads_started":1}
I20260809 01:03:44.229684 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:44.228915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:35945)
0809 01:03:44.229388 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:44.229470 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":229,"threads_started":1}
I20260809 01:03:44.313102 22582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:44.312450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:35933)
0809 01:03:44.312883 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:44.312956 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":222,"threads_started":1}
I20260809 01:03:44.767365 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:44.766650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:52819)
0809 01:03:44.767081 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:44.767157 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":214,"threads_started":1}
I20260809 01:03:45.231221 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:45.230487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:34363)
0809 01:03:45.230929 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:45.231007 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":226,"threads_started":1}
I20260809 01:03:45.314687 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:45.313985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:54139)
0809 01:03:45.314423 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:45.314507 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":232,"threads_started":1}
I20260809 01:03:45.768805 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:45.768095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:50801)
0809 01:03:45.768524 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:45.768612 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":229,"threads_started":1}
I20260809 01:03:46.232678 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:46.232031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:47161)
0809 01:03:46.232437 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:46.232510 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":226,"threads_started":1}
I20260809 01:03:46.316380 22588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:46.315661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:35849)
0809 01:03:46.316094 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:46.316174 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":217,"threads_started":1}
I20260809 01:03:46.770314 22589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:46.769631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:52727)
0809 01:03:46.770040 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:46.770115 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":206,"threads_started":1}
I20260809 01:03:47.234393 22590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:47.233639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:44621)
0809 01:03:47.234113 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:47.234214 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":240,"threads_started":1}
I20260809 01:03:47.318138 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:47.317320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:34803)
0809 01:03:47.317869 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:47.317945 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":318,"threads_started":1}
I20260809 01:03:47.772277 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:47.771433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:46205)
0809 01:03:47.772018 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:47.772099 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":288,"threads_started":1}
I20260809 01:03:48.236020 22593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:48.235333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:39863)
0809 01:03:48.235776 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:48.235853 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":213,"threads_started":1}
I20260809 01:03:48.319850 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:48.319065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:41877)
0809 01:03:48.319576 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:48.319670 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":302,"threads_started":1}
I20260809 01:03:48.773813 22595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:48.773043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:47915)
0809 01:03:48.773563 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:48.773654 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":311,"threads_started":1}
I20260809 01:03:49.237537 22596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:49.236861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:48307)
0809 01:03:49.237294 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:49.237381 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":224,"threads_started":1}
I20260809 01:03:49.321518 22597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:49.320702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:54129)
0809 01:03:49.321202 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:49.321301 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":211,"threads_started":1}
I20260809 01:03:49.775703 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:49.774827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:56549)
0809 01:03:49.775360 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:49.775470 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":220,"threads_started":1}
I20260809 01:03:50.239212 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:50.238443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:47507)
0809 01:03:50.238916 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:50.238995 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":222,"threads_started":1}
I20260809 01:03:50.322994 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:50.322323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:55239)
0809 01:03:50.322756 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:50.322836 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":244,"threads_started":1}
I20260809 01:03:50.777227 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:50.776543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:33383)
0809 01:03:50.776985 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:50.777074 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":244,"threads_started":1}
I20260809 01:03:51.240756 22603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:51.240066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:54037)
0809 01:03:51.240505 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:51.240595 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":223,"threads_started":1}
I20260809 01:03:51.324422 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:51.323748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:56623)
0809 01:03:51.324180 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:51.324255 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":213,"threads_started":1}
I20260809 01:03:51.778906 22605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:51.778199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:57991)
0809 01:03:51.778657 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:51.778739 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":230,"threads_started":1}
I20260809 01:03:52.242358 22606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:52.241573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:44971)
0809 01:03:52.242037 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:52.242125 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":232,"threads_started":1}
I20260809 01:03:52.325968 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:52.325251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:40513)
0809 01:03:52.325690 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:52.325773 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":236,"threads_started":1}
I20260809 01:03:52.780418 22608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:52.779772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:37269)
0809 01:03:52.780185 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:52.780258 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":230,"threads_started":1}
I20260809 01:03:53.244238 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:53.243269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:58651)
0809 01:03:53.243892 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:53.243992 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":351,"threads_started":1}
I20260809 01:03:53.327737 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:53.326867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:53817)
0809 01:03:53.327456 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:53.327550 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":297,"threads_started":1}
I20260809 01:03:53.781984 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:53.781285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:45543)
0809 01:03:53.781713 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:53.781808 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":206,"threads_started":1}
I20260809 01:03:54.245970 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:54.245168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:37005)
0809 01:03:54.245644 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:54.245767 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":232,"threads_started":1}
I20260809 01:03:54.329304 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:54.328620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:41881)
0809 01:03:54.329063 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:54.329152 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":229,"threads_started":1}
I20260809 01:03:54.783703 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:54.782905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:35279)
0809 01:03:54.783411 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:54.783521 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":235,"threads_started":1}
I20260809 01:03:55.247543 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:55.246811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:57419)
0809 01:03:55.247268 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:55.247390 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":205,"threads_started":1}
I20260809 01:03:55.331103 22616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:55.330300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:56991)
0809 01:03:55.330826 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:55.330917 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":288,"threads_started":1}
I20260809 01:03:55.785248 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:55.784528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:54025)
0809 01:03:55.784962 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:55.785051 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":209,"threads_started":1}
I20260809 01:03:56.249064 22619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:56.248341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:59443)
0809 01:03:56.248796 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:56.248885 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":240,"threads_started":1}
I20260809 01:03:56.332521 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:56.331871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:41013)
0809 01:03:56.332271 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:56.332344 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":212,"threads_started":1}
I20260809 01:03:56.786906 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:56.786217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:51723)
0809 01:03:56.786642 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:56.786733 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":222,"threads_started":1}
I20260809 01:03:57.250741 22623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:57.249901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:55397)
0809 01:03:57.250410 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:57.250490 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":279,"threads_started":1}
I20260809 01:03:57.334342 22624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:57.333613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:49015)
0809 01:03:57.334052 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:57.334164 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":231,"threads_started":1}
I20260809 01:03:57.788637 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:57.787894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.130:46635)
0809 01:03:57.788330 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:57.788443 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":209,"threads_started":1}
I20260809 01:03:58.234047 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 22120
I20260809 01:03:58.252607 22627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:58.251830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.131:49169)
0809 01:03:58.252310 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:58.252428 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":261,"threads_started":1}
I20260809 01:03:58.260638 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:58.235221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52296 (local address 127.4.82.129:41329)
0809 01:03:58.235835 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:58.235853 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:58.236020 (+ 167us) server_negotiation.cc:407] Connection header received
0809 01:03:58.237658 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:58.237667 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:58.238079 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:58.238426 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:03:58.240040 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:58.241677 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:58.243262 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:58.244114 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:58.244321 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:58.253873 (+ 9552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:03:58.253882 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:03:58.253898 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:03:58.253936 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:03:58.257003 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:58.257793 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:58.257803 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:58.257812 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:58.258012 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:58.258462 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:58.258467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:58.258471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:58.259011 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:03:58.259397 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:58.259904 (+ 507us) server_negotiation.cc:299] Negotiation successful
0809 01:03:58.260340 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":315,"threads_started":1}
I20260809 01:03:58.262032 22286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:03:58.352130 22629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:58.350703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34755 (local address 127.4.82.129:59847)
0809 01:03:58.351489 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0809 01:03:58.351741 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.190:34755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":425,"threads_started":1}
I20260809 01:03:58.355463 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 22120
I20260809 01:03:58.377805 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 22294
I20260809 01:03:58.395625 22630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:58.378781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44598 (local address 127.4.82.130:44101)
0809 01:03:58.379368 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:58.379392 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:58.379430 (+ 38us) server_negotiation.cc:407] Connection header received
0809 01:03:58.380968 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:58.380978 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:58.381389 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:58.381696 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:03:58.383076 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:58.384374 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:58.385665 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:58.386293 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:58.386425 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:58.389244 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:03:58.389252 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:03:58.389265 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:03:58.389304 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:03:58.392299 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:58.393047 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:58.393053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:58.393058 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:58.393211 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:58.393637 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:58.393640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:58.393642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:58.394097 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:03:58.394317 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:58.394726 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 01:03:58.395151 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":248,"threads_started":1}
I20260809 01:03:58.396940 22430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:03:58.479197 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 22294
I20260809 01:03:58.499081 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 22438
I20260809 01:03:58.516243 22632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:03:58.500172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58292 (local address 127.4.82.131:39695)
0809 01:03:58.500759 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0809 01:03:58.500774 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 01:03:58.500815 (+ 41us) server_negotiation.cc:407] Connection header received
0809 01:03:58.502409 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:03:58.502419 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:03:58.502839 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:03:58.503112 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:03:58.504432 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:58.505618 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:58.506817 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:03:58.507513 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:03:58.507636 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:03:58.510134 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:03:58.510141 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:03:58.510154 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:03:58.510195 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:03:58.513385 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:58.513998 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:58.514003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:58.514009 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:58.514158 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:03:58.514481 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:03:58.514484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:03:58.514486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:03:58.514970 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:03:58.515176 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 01:03:58.515550 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 01:03:58.515953 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":268,"threads_started":1}
I20260809 01:03:58.517550 22569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:03:58.601583 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 22438
2026-08-09T01:03:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64395 ms)
[----------] 30 tests from SecurityITest (294394 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:03:58 dist-test-slave-07c2 krb5kdc[22637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:03:58 dist-test-slave-07c2 krb5kdc[22637](info): set up 2 sockets
Aug 09 01:03:58 dist-test-slave-07c2 krb5kdc[22637](info): commencing operation
krb5kdc: starting...
W20260809 01:03:59.755921 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:03:59 dist-test-slave-07c2 krb5kdc[22637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:03:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:03:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:04:00.052667 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:37211
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:36925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:37211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:00.387131 22650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:00.387425 22650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:00.387610 22650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:00.396148 22650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:00.396308 22650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:00.396389 22650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:00.396505 22650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:00.396579 22650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:00.412640 22650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:37211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:37211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:00.414729 22650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:00.416479 22650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:00.430974 22657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:00.431178 22659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:00.431529 22656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:00.432762 22650 server_base.cc:1034] running on GCE node
I20260809 01:04:00.433606 22650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:00.435019 22650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:00.436283 22650 hybrid_clock.cc:648] HybridClock initialized: now 1786237440436192 us; error 72 us; skew 500 ppm
I20260809 01:04:00.436657 22650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:00 dist-test-slave-07c2 krb5kdc[22637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237440, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:00.440160 22650 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:04:00.441666 22650 webserver.cc:460] Webserver started at http://127.4.82.190:42423/ using document root <none> and password file <none>
I20260809 01:04:00.442342 22650 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:00.442473 22650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:00.442812 22650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:00.445860 22650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "ad472a1779964d218579c51368deee48"
format_stamp: "Formatted at 2026-08-09 01:04:00 on dist-test-slave-07c2"
server_key: "ecc850d9c634f9a38fde80386951550b"
server_key_iv: "0e9c335b4740aa47b47cf303560a098d"
server_key_version: "encryptionkey@0"
I20260809 01:04:00.446621 22650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "ad472a1779964d218579c51368deee48"
format_stamp: "Formatted at 2026-08-09 01:04:00 on dist-test-slave-07c2"
server_key: "ecc850d9c634f9a38fde80386951550b"
server_key_iv: "0e9c335b4740aa47b47cf303560a098d"
server_key_version: "encryptionkey@0"
I20260809 01:04:00.451781 22650 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 01:04:00.455176 22666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:00.456543 22650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 01:04:00.456748 22650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "ad472a1779964d218579c51368deee48"
format_stamp: "Formatted at 2026-08-09 01:04:00 on dist-test-slave-07c2"
server_key: "ecc850d9c634f9a38fde80386951550b"
server_key_iv: "0e9c335b4740aa47b47cf303560a098d"
server_key_version: "encryptionkey@0"
I20260809 01:04:00.456947 22650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:00.486375 22650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:00.490348 22650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:00.490653 22650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:00.509325 22650 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:37211
I20260809 01:04:00.509362 22720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:37211 every 8 connection(s)
I20260809 01:04:00.510699 22650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:00.514415 22721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:00.520339 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 22650
I20260809 01:04:00.520612 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:00.521303 4426 external_mini_cluster.cc:1428] Setting key c6e27af3ec1ed389a5f4aa12437b7f21
I20260809 01:04:00.525974 22721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48: Bootstrap starting.
I20260809 01:04:00.529838 22721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:00.531095 22721 log.cc:824] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:00.534380 22721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48: No bootstrap required, opened a new log
Aug 09 01:04:00 dist-test-slave-07c2 krb5kdc[22637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:00.544510 22721 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad472a1779964d218579c51368deee48" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37211 } }
I20260809 01:04:00.545034 22721 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:00.545145 22721 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad472a1779964d218579c51368deee48, State: Initialized, Role: FOLLOWER
I20260809 01:04:00.545856 22721 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad472a1779964d218579c51368deee48" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37211 } }
I20260809 01:04:00.546197 22721 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:00.546363 22721 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:00.546543 22721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:00.548249 22724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:00.523585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44134 (local address 127.4.82.190:37211)
0809 01:04:00.524406 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:00.524423 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:00.524479 (+ 56us) server_negotiation.cc:407] Connection header received
0809 01:04:00.526923 (+ 2444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:00.526949 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:00.527459 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:00.527929 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:00.529322 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:00.531361 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:00.533169 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:00.534219 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:00.534423 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:00.538977 (+ 4554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:00.538989 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:00.539010 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:00.539065 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:00.543332 (+ 4267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:00.544074 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:00.544083 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:00.544093 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:00.544326 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:00.544705 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:00.544709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:00.544712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:00.545498 (+ 786us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:00.545812 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:00.546272 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 01:04:00.546758 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":366,"threads_started":1}
I20260809 01:04:00.549564 22721 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad472a1779964d218579c51368deee48" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37211 } }
I20260809 01:04:00.550083 22721 leader_election.cc:302] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad472a1779964d218579c51368deee48; no voters:
I20260809 01:04:00.550803 22721 leader_election.cc:288] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:00.550920 22726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:00.551868 22726 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [term 1 LEADER]: Becoming Leader. State: Replica: ad472a1779964d218579c51368deee48, State: Running, Role: LEADER
I20260809 01:04:00.552423 22726 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad472a1779964d218579c51368deee48" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37211 } }
I20260809 01:04:00.553361 22721 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:00.555527 22728 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad472a1779964d218579c51368deee48. Latest consensus state: current_term: 1 leader_uuid: "ad472a1779964d218579c51368deee48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad472a1779964d218579c51368deee48" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37211 } } }
I20260809 01:04:00.556018 22728 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:00.555554 22727 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad472a1779964d218579c51368deee48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad472a1779964d218579c51368deee48" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37211 } } }
I20260809 01:04:00.567975 22727 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:00.569043 22733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:00.573549 22733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:00.581696 22733 catalog_manager.cc:1347] Generated new cluster ID: 693bb9cca652419d9e3e9b535a50dab8
I20260809 01:04:00.581828 22733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:00.591889 22733 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:00.593721 22733 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:00.607632 22733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48: Generated new TSK 0
I20260809 01:04:00.608737 22733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 01:04:00.671115 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37211
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:36925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:01.003679 22748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:01.003925 22748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:01.004112 22748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:01.011106 22748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:01.011211 22748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:01.011435 22748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:04:01.020823 22748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37211
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:01.022034 22748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:01.023368 22748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:01.036144 22753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:01.036317 22756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:01.036401 22754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:01.038579 22748 server_base.cc:1034] running on GCE node
I20260809 01:04:01.039172 22748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:01.040122 22748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:01.041363 22748 hybrid_clock.cc:648] HybridClock initialized: now 1786237441041327 us; error 47 us; skew 500 ppm
I20260809 01:04:01.041678 22748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:01 dist-test-slave-07c2 krb5kdc[22637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237441, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:01.044857 22748 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:04:01.046240 22748 webserver.cc:460] Webserver started at http://127.4.82.129:38221/ using document root <none> and password file <none>
I20260809 01:04:01.046924 22748 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:01.047073 22748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:01.047456 22748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:01.050321 22748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "a26892ca9161423e8b4eb52c0098f898"
format_stamp: "Formatted at 2026-08-09 01:04:01 on dist-test-slave-07c2"
server_key: "41f20bcbffb9a45eaa63076ef326b8ef"
server_key_iv: "11c0bba6529d87449534a04e932f1e89"
server_key_version: "encryptionkey@0"
I20260809 01:04:01.051120 22748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "a26892ca9161423e8b4eb52c0098f898"
format_stamp: "Formatted at 2026-08-09 01:04:01 on dist-test-slave-07c2"
server_key: "41f20bcbffb9a45eaa63076ef326b8ef"
server_key_iv: "11c0bba6529d87449534a04e932f1e89"
server_key_version: "encryptionkey@0"
I20260809 01:04:01.056336 22748 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 01:04:01.059697 22763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:01.061096 22748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 01:04:01.061311 22748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "a26892ca9161423e8b4eb52c0098f898"
format_stamp: "Formatted at 2026-08-09 01:04:01 on dist-test-slave-07c2"
server_key: "41f20bcbffb9a45eaa63076ef326b8ef"
server_key_iv: "11c0bba6529d87449534a04e932f1e89"
server_key_version: "encryptionkey@0"
I20260809 01:04:01.061546 22748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:01.099210 22748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:01.103459 22748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:01.103775 22748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:01.104789 22748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:01.106130 22748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:01.106252 22748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:01.106422 22748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:01.106501 22748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:01.140468 22748 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:44177
I20260809 01:04:01.140563 22880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:44177 every 8 connection(s)
I20260809 01:04:01.141856 22748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:04:01.150090 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 22748
I20260809 01:04:01.150329 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:04:01.151010 4426 external_mini_cluster.cc:1428] Setting key 6bd821e1d5938e7480492d44d90c92c5
Aug 09 01:04:01 dist-test-slave-07c2 krb5kdc[22637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237441, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:01.163071 22884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:01.144597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:39941 (local address 127.4.82.190:37211)
0809 01:04:01.145073 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:01.145077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:01.146046 (+ 969us) server_negotiation.cc:407] Connection header received
0809 01:04:01.147981 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:01.147987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:01.148089 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:01.148214 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:01.149748 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:01.150576 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:01.152413 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:01.152913 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:01.153031 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:01.157210 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:01.157215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:01.157219 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:01.157249 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:01.159864 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:01.160683 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:01.160691 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:01.160699 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:01.160816 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:01.161318 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:01.161323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:01.161328 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:01.161608 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:01.161770 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:01.162469 (+ 699us) server_negotiation.cc:299] Negotiation successful
0809 01:04:01.162697 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":234,"threads_started":1}
I20260809 01:04:01.164074 22885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:01.145157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:37211 (local address 127.4.82.129:39941)
0809 01:04:01.145885 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:01.145932 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:01.147714 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:01.148452 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:01.148460 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:01.148991 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:01.149540 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:01.149558 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:01.150728 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:01.150740 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:01.152272 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:01.152286 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:01.153163 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:01.153172 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:01.153472 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:01.154267 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:04:01.154307 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:01.156978 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:01.160034 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:01.160045 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:01.160062 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:01.160553 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:01.160935 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:01.160940 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:01.160944 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:01.161189 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:01.161744 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:01.161752 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:01.162089 (+ 337us) client_negotiation.cc:769] Sending connection context
0809 01:04:01.162354 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 01:04:01.162882 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":288,"threads_started":1}
I20260809 01:04:01.165582 22881 heartbeater.cc:347] Connected to a master server at 127.4.82.190:37211
I20260809 01:04:01.165982 22881 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:01.166904 22881 heartbeater.cc:511] Master 127.4.82.190:37211 requested a full tablet report, sending...
I20260809 01:04:01.169045 22685 ts_manager.cc:194] Registered new tserver with Master: a26892ca9161423e8b4eb52c0098f898 (127.4.82.129:44177)
I20260809 01:04:01.170461 22685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:39941
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 01:04:01.214421 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37211
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:36925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:01.543881 22888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:01.544147 22888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:01.544314 22888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:01.551530 22888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:01.551638 22888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:01.551759 22888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:04:01.561832 22888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37211
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:01.563043 22888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:01.564452 22888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:01.579772 22894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:01.582954 22893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:01.583362 22896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:01.583927 22888 server_base.cc:1034] running on GCE node
I20260809 01:04:01.584781 22888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:01.586292 22888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:01.587605 22888 hybrid_clock.cc:648] HybridClock initialized: now 1786237441587526 us; error 74 us; skew 500 ppm
I20260809 01:04:01.588029 22888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:01 dist-test-slave-07c2 krb5kdc[22637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237441, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:01.591346 22888 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:04:01.592880 22888 webserver.cc:460] Webserver started at http://127.4.82.130:43769/ using document root <none> and password file <none>
I20260809 01:04:01.593578 22888 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:01.593711 22888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:01.594023 22888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:01.596949 22888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "be5afc9ceff74e7fbd402fe58cc0953e"
format_stamp: "Formatted at 2026-08-09 01:04:01 on dist-test-slave-07c2"
server_key: "d22d0003fb7a7283d0d4d3a6089bd8f4"
server_key_iv: "31b1195f4870f96fffce3bb1c7ac7d8c"
server_key_version: "encryptionkey@0"
I20260809 01:04:01.597734 22888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "be5afc9ceff74e7fbd402fe58cc0953e"
format_stamp: "Formatted at 2026-08-09 01:04:01 on dist-test-slave-07c2"
server_key: "d22d0003fb7a7283d0d4d3a6089bd8f4"
server_key_iv: "31b1195f4870f96fffce3bb1c7ac7d8c"
server_key_version: "encryptionkey@0"
I20260809 01:04:01.602754 22888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 01:04:01.606057 22903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:01.607805 22888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:04:01.607975 22888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "be5afc9ceff74e7fbd402fe58cc0953e"
format_stamp: "Formatted at 2026-08-09 01:04:01 on dist-test-slave-07c2"
server_key: "d22d0003fb7a7283d0d4d3a6089bd8f4"
server_key_iv: "31b1195f4870f96fffce3bb1c7ac7d8c"
server_key_version: "encryptionkey@0"
I20260809 01:04:01.608161 22888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:01.625578 22888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:01.629703 22888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:01.629931 22888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:01.630807 22888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:01.632192 22888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:01.632263 22888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:01.632349 22888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:01.632398 22888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:01.665630 22888 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:46509
I20260809 01:04:01.665704 23023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:46509 every 8 connection(s)
I20260809 01:04:01.667004 22888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:04:01.673803 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 22888
I20260809 01:04:01.674011 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:04:01.674528 4426 external_mini_cluster.cc:1428] Setting key f8072a29d15058a9fafef98c22b1f2de
Aug 09 01:04:01 dist-test-slave-07c2 krb5kdc[22637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237441, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:01.688571 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:01.669675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:56709 (local address 127.4.82.190:37211)
0809 01:04:01.670135 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:01.670140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:01.671130 (+ 990us) server_negotiation.cc:407] Connection header received
0809 01:04:01.673000 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:01.673005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:01.673089 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:01.673225 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:01.674842 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:01.675873 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:01.678147 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:01.678564 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:01.678657 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:01.682749 (+ 4092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:01.682755 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:01.682758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:01.682788 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:01.685472 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:01.686181 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:01.686187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:01.686190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:01.686333 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:01.686766 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:01.686771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:01.686774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:01.687036 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:01.687209 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:01.687936 (+ 727us) server_negotiation.cc:299] Negotiation successful
0809 01:04:01.688171 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":229,"threads_started":1}
I20260809 01:04:01.689502 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:01.670275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:37211 (local address 127.4.82.130:56709)
0809 01:04:01.670966 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:01.671015 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:01.672746 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:01.673469 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:01.673477 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:01.674050 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:01.674664 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:01.674693 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:01.676023 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:01.676027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:01.677977 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:01.677997 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:01.678808 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:01.678818 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:01.679139 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:01.679888 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:04:01.679927 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:01.682511 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:01.685610 (+ 3099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:01.685619 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:01.685634 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:01.686058 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:01.686447 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:01.686452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:01.686456 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:01.686649 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:01.687186 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:01.687194 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:01.687587 (+ 393us) client_negotiation.cc:769] Sending connection context
0809 01:04:01.687860 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 01:04:01.688384 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":272,"threads_started":1}
I20260809 01:04:01.690984 23024 heartbeater.cc:347] Connected to a master server at 127.4.82.190:37211
I20260809 01:04:01.691278 23024 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:01.692030 23024 heartbeater.cc:511] Master 127.4.82.190:37211 requested a full tablet report, sending...
I20260809 01:04:01.693476 22685 ts_manager.cc:194] Registered new tserver with Master: be5afc9ceff74e7fbd402fe58cc0953e (127.4.82.130:46509)
I20260809 01:04:01.694260 22685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:56709
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 01:04:01.736261 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37211
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:36925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:02.062299 23030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:02.062575 23030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:02.062788 23030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:02.070485 23030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:02.070593 23030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:02.070818 23030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:04:02.080659 23030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:37211
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:02.081979 23030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:02.083231 23030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:02.096201 23036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:02.096267 23039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:02.096243 23037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:02.097203 23030 server_base.cc:1034] running on GCE node
I20260809 01:04:02.097750 23030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:02.099272 23030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:02.100531 23030 hybrid_clock.cc:648] HybridClock initialized: now 1786237442100498 us; error 27 us; skew 500 ppm
I20260809 01:04:02.100862 23030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:02 dist-test-slave-07c2 krb5kdc[22637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:02.104123 23030 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:04:02.105535 23030 webserver.cc:460] Webserver started at http://127.4.82.131:45881/ using document root <none> and password file <none>
I20260809 01:04:02.106253 23030 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:02.106395 23030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:02.106724 23030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:02.109652 23030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "4a4fbd3839f34ff1957c78b4e23d5f98"
format_stamp: "Formatted at 2026-08-09 01:04:02 on dist-test-slave-07c2"
server_key: "9fed6ceab4326170ac304e84810245db"
server_key_iv: "9314aa979e032041d4ce5a6ec5acd5c7"
server_key_version: "encryptionkey@0"
I20260809 01:04:02.110502 23030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "4a4fbd3839f34ff1957c78b4e23d5f98"
format_stamp: "Formatted at 2026-08-09 01:04:02 on dist-test-slave-07c2"
server_key: "9fed6ceab4326170ac304e84810245db"
server_key_iv: "9314aa979e032041d4ce5a6ec5acd5c7"
server_key_version: "encryptionkey@0"
I20260809 01:04:02.115430 23030 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 01:04:02.118592 23046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:02.119871 23030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:04:02.120040 23030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "4a4fbd3839f34ff1957c78b4e23d5f98"
format_stamp: "Formatted at 2026-08-09 01:04:02 on dist-test-slave-07c2"
server_key: "9fed6ceab4326170ac304e84810245db"
server_key_iv: "9314aa979e032041d4ce5a6ec5acd5c7"
server_key_version: "encryptionkey@0"
I20260809 01:04:02.120234 23030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:02.136982 23030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:02.141106 23030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:02.141434 23030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:02.142490 23030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:02.144174 23030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:02.144316 23030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 01:04:02.144449 23030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:02.144506 23030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:02.174669 22881 heartbeater.cc:503] Master 127.4.82.190:37211 was elected leader, sending a full tablet report...
I20260809 01:04:02.179261 23030 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:34789
I20260809 01:04:02.179344 23165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:34789 every 8 connection(s)
I20260809 01:04:02.180526 23030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:04:02.181870 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 23030
I20260809 01:04:02.182065 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:04:02.182518 4426 external_mini_cluster.cc:1428] Setting key b5c746c09e184b5a861a64aeab286ff1
Aug 09 01:04:02 dist-test-slave-07c2 krb5kdc[22637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237442, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:02.199549 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.183151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:50713 (local address 127.4.82.190:37211)
0809 01:04:02.183374 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:02.183380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:02.184753 (+ 1373us) server_negotiation.cc:407] Connection header received
0809 01:04:02.186514 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:02.186517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:02.186581 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:02.186660 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:02.188122 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.188855 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.190278 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.190660 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.190760 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.193985 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:02.193991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:02.193994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:02.194022 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:02.196743 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:02.197422 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:02.197429 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:02.197437 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:02.197548 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:02.197966 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:02.197973 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:02.197980 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:02.198236 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:02.198398 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:02.198987 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 01:04:02.199200 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 01:04:02.200353 23168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.183792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:37211 (local address 127.4.82.131:50713)
0809 01:04:02.184588 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:02.184632 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:02.186286 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:02.186857 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:02.186864 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:02.187411 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:02.187987 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:02.188003 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.188982 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.188985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:02.190144 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:02.190154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.190846 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.190850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:02.191026 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.191634 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:04:02.191659 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:02.193772 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:02.196877 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:02.196887 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:02.196904 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:02.197292 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:02.197660 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:02.197665 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:02.197668 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:02.197844 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:02.198360 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:02.198368 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:02.198645 (+ 277us) client_negotiation.cc:769] Sending connection context
0809 01:04:02.198884 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 01:04:02.199367 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":355,"threads_started":1}
I20260809 01:04:02.201489 23166 heartbeater.cc:347] Connected to a master server at 127.4.82.190:37211
I20260809 01:04:02.201790 23166 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:02.202456 23166 heartbeater.cc:511] Master 127.4.82.190:37211 requested a full tablet report, sending...
I20260809 01:04:02.203852 22685 ts_manager.cc:194] Registered new tserver with Master: 4a4fbd3839f34ff1957c78b4e23d5f98 (127.4.82.131:34789)
I20260809 01:04:02.204622 22685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:50713
I20260809 01:04:02.209406 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:04:02.225942 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.213958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44144 (local address 127.4.82.190:37211)
0809 01:04:02.214173 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:02.214179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:02.214457 (+ 278us) server_negotiation.cc:407] Connection header received
0809 01:04:02.214579 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:02.214582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:02.214643 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:02.214726 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:02.216144 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.216835 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.218071 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.218465 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.218552 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.220840 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:02.220845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:02.220848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:02.220875 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:02.223160 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:02.223853 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:02.223857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:02.223859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:02.223966 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:02.224381 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:02.224385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:02.224387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:02.224738 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:02.225029 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:02.225406 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 01:04:02.225617 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 01:04:02.229691 22685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:04:02.236739 22685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:04:02.271525 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.248864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42858 (local address 127.4.82.131:34789)
0809 01:04:02.264030 (+ 15166us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:02.264047 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:02.264078 (+ 31us) server_negotiation.cc:407] Connection header received
0809 01:04:02.264170 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:02.264178 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:02.264384 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:02.264568 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:02.265107 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.266783 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.269006 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.269970 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.270226 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.270458 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:02.270983 (+ 525us) server_negotiation.cc:299] Negotiation successful
0809 01:04:02.271145 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14995,"thread_start_us":14832,"threads_started":1}
I20260809 01:04:02.271533 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.248357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:34789 (local address 127.0.0.1:42858)
0809 01:04:02.260431 (+ 12074us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:02.260453 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:02.260597 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:02.264583 (+ 3986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:02.264588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:02.264613 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:02.264926 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:02.264936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.266903 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.266912 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:02.268862 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:02.268872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.270353 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.270367 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:02.270648 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.270706 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 01:04:02.270839 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 01:04:02.271020 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11911,"mutex_wait_us":36,"thread_start_us":12489,"threads_started":1}
I20260809 01:04:02.271574 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.251204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37236 (local address 127.4.82.129:44177)
0809 01:04:02.262454 (+ 11250us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:02.262462 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:02.262488 (+ 26us) server_negotiation.cc:407] Connection header received
0809 01:04:02.262602 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:02.262611 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:02.262878 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:02.263084 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:02.263992 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.265740 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.268247 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.269712 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.269872 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.270111 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:02.271032 (+ 921us) server_negotiation.cc:299] Negotiation successful
0809 01:04:02.271176 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11066,"thread_start_us":10901,"threads_started":1}
I20260809 01:04:02.272150 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.248413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60682 (local address 127.4.82.130:46509)
0809 01:04:02.262047 (+ 13634us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:02.262060 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:02.262494 (+ 434us) server_negotiation.cc:407] Connection header received
0809 01:04:02.262635 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:02.262646 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:02.262912 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:02.263254 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:02.264088 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.265524 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.268436 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.269797 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.269912 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.270087 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:02.271651 (+ 1564us) server_negotiation.cc:299] Negotiation successful
0809 01:04:02.271815 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13424,"thread_start_us":13303,"threads_started":1}
I20260809 01:04:02.272146 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.248249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:46509 (local address 127.0.0.1:60682)
0809 01:04:02.258888 (+ 10639us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:02.258925 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:02.259111 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:02.263408 (+ 4297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:02.263413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:02.263440 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:02.263943 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:02.263956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.265642 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.265649 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:02.268281 (+ 2632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:02.268296 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.270385 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.270392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:02.271104 (+ 712us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.271133 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 01:04:02.271213 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 01:04:02.271345 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10384,"mutex_wait_us":486,"thread_start_us":10310,"threads_started":1}
I20260809 01:04:02.272709 23178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.250443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:44177 (local address 127.0.0.1:37236)
0809 01:04:02.261175 (+ 10732us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:02.261251 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:02.261422 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:02.263181 (+ 1759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:02.263190 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:02.263233 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:02.263752 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:02.263769 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.265905 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.265910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:02.268093 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:02.268108 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.270660 (+ 2552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.270667 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:02.270838 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.270868 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 01:04:02.270936 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 01:04:02.271031 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10588,"mutex_wait_us":36,"thread_start_us":8026,"threads_started":1}
I20260809 01:04:02.275748 22815 tablet_service.cc:1467] Processing CreateTablet for tablet 80772a56a3c14c2eb0ff39c9a3abfdc2 (DEFAULT_TABLE table=test-table [id=63db594ea0874d818b680d85774bc90b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:02.275748 23098 tablet_service.cc:1467] Processing CreateTablet for tablet 80772a56a3c14c2eb0ff39c9a3abfdc2 (DEFAULT_TABLE table=test-table [id=63db594ea0874d818b680d85774bc90b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:02.275755 22952 tablet_service.cc:1467] Processing CreateTablet for tablet 80772a56a3c14c2eb0ff39c9a3abfdc2 (DEFAULT_TABLE table=test-table [id=63db594ea0874d818b680d85774bc90b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:02.276989 22815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80772a56a3c14c2eb0ff39c9a3abfdc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:02.277055 22952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80772a56a3c14c2eb0ff39c9a3abfdc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:02.277052 23098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80772a56a3c14c2eb0ff39c9a3abfdc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:02.288911 23185 tablet_bootstrap.cc:492] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898: Bootstrap starting.
I20260809 01:04:02.291190 23186 tablet_bootstrap.cc:492] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e: Bootstrap starting.
I20260809 01:04:02.292313 23187 tablet_bootstrap.cc:492] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98: Bootstrap starting.
I20260809 01:04:02.292652 23185 tablet_bootstrap.cc:654] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:02.293937 23185 log.cc:824] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:02.295913 23187 tablet_bootstrap.cc:654] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:02.296880 23185 tablet_bootstrap.cc:492] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898: No bootstrap required, opened a new log
I20260809 01:04:02.297184 23185 ts_tablet_manager.cc:1397] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260809 01:04:02.297163 23187 log.cc:824] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:02.298448 23186 tablet_bootstrap.cc:654] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:02.299693 23186 log.cc:824] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:02.300330 23187 tablet_bootstrap.cc:492] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98: No bootstrap required, opened a new log
I20260809 01:04:02.300603 23187 ts_tablet_manager.cc:1397] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 01:04:02.302651 23186 tablet_bootstrap.cc:492] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e: No bootstrap required, opened a new log
I20260809 01:04:02.302911 23186 ts_tablet_manager.cc:1397] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260809 01:04:02.305451 23185 raft_consensus.cc:348] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be5afc9ceff74e7fbd402fe58cc0953e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 46509 } } peers { permanent_uuid: "4a4fbd3839f34ff1957c78b4e23d5f98" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34789 } } peers { permanent_uuid: "a26892ca9161423e8b4eb52c0098f898" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44177 } }
I20260809 01:04:02.305851 23185 raft_consensus.cc:374] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:02.305974 23185 raft_consensus.cc:729] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a26892ca9161423e8b4eb52c0098f898, State: Initialized, Role: FOLLOWER
I20260809 01:04:02.306576 23185 consensus_queue.cc:260] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be5afc9ceff74e7fbd402fe58cc0953e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 46509 } } peers { permanent_uuid: "4a4fbd3839f34ff1957c78b4e23d5f98" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34789 } } peers { permanent_uuid: "a26892ca9161423e8b4eb52c0098f898" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44177 } }
I20260809 01:04:02.311556 23187 raft_consensus.cc:348] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be5afc9ceff74e7fbd402fe58cc0953e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 46509 } } peers { permanent_uuid: "4a4fbd3839f34ff1957c78b4e23d5f98" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34789 } } peers { permanent_uuid: "a26892ca9161423e8b4eb52c0098f898" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44177 } }
I20260809 01:04:02.311901 23185 ts_tablet_manager.cc:1428] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898: Time spent starting tablet: real 0.015s user 0.003s sys 0.011s
I20260809 01:04:02.312201 23187 raft_consensus.cc:374] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:02.312366 23187 raft_consensus.cc:729] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a4fbd3839f34ff1957c78b4e23d5f98, State: Initialized, Role: FOLLOWER
I20260809 01:04:02.313014 23187 consensus_queue.cc:260] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be5afc9ceff74e7fbd402fe58cc0953e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 46509 } } peers { permanent_uuid: "4a4fbd3839f34ff1957c78b4e23d5f98" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34789 } } peers { permanent_uuid: "a26892ca9161423e8b4eb52c0098f898" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44177 } }
I20260809 01:04:02.314100 23166 heartbeater.cc:503] Master 127.4.82.190:37211 was elected leader, sending a full tablet report...
I20260809 01:04:02.313791 23186 raft_consensus.cc:348] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be5afc9ceff74e7fbd402fe58cc0953e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 46509 } } peers { permanent_uuid: "4a4fbd3839f34ff1957c78b4e23d5f98" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34789 } } peers { permanent_uuid: "a26892ca9161423e8b4eb52c0098f898" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44177 } }
I20260809 01:04:02.314257 23186 raft_consensus.cc:374] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:02.314368 23186 raft_consensus.cc:729] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be5afc9ceff74e7fbd402fe58cc0953e, State: Initialized, Role: FOLLOWER
I20260809 01:04:02.314914 23186 consensus_queue.cc:260] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be5afc9ceff74e7fbd402fe58cc0953e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 46509 } } peers { permanent_uuid: "4a4fbd3839f34ff1957c78b4e23d5f98" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34789 } } peers { permanent_uuid: "a26892ca9161423e8b4eb52c0098f898" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44177 } }
I20260809 01:04:02.317531 23187 ts_tablet_manager.cc:1428] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
I20260809 01:04:02.320133 23024 heartbeater.cc:503] Master 127.4.82.190:37211 was elected leader, sending a full tablet report...
I20260809 01:04:02.320603 23186 ts_tablet_manager.cc:1428] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e: Time spent starting tablet: real 0.018s user 0.018s sys 0.000s
W20260809 01:04:02.394286 22883 tablet.cc:2369] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 01:04:02.419170 23025 tablet.cc:2369] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 01:04:02.432269 23167 tablet.cc:2369] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:04:02.488420 23191 raft_consensus.cc:482] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:04:02.488776 23191 raft_consensus.cc:504] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be5afc9ceff74e7fbd402fe58cc0953e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 46509 } } peers { permanent_uuid: "4a4fbd3839f34ff1957c78b4e23d5f98" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34789 } } peers { permanent_uuid: "a26892ca9161423e8b4eb52c0098f898" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44177 } }
I20260809 01:04:02.490509 23191 leader_election.cc:288] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be5afc9ceff74e7fbd402fe58cc0953e (127.4.82.130:46509), 4a4fbd3839f34ff1957c78b4e23d5f98 (127.4.82.131:34789)
I20260809 01:04:02.497591 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.491247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:34167 (local address 127.4.82.130:46509)
0809 01:04:02.491475 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:02.491481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:02.491720 (+ 239us) server_negotiation.cc:407] Connection header received
0809 01:04:02.491941 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:02.491948 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:02.492039 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:02.492134 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:02.492676 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.493847 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.495677 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.496637 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.496749 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.496798 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:02.497193 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 01:04:02.497319 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 01:04:02.497749 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.490793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:46509 (local address 127.4.82.129:34167)
0809 01:04:02.491567 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:02.491593 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:02.491781 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:02.492151 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:02.492155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:02.492184 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:02.492516 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:02.492528 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.493983 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.493987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:02.495527 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:02.495539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.496843 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.496849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:02.497024 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.497060 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 01:04:02.497167 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 01:04:02.497308 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":19,"spinlock_wait_cycles":21248,"thread_start_us":534,"threads_started":1}
I20260809 01:04:02.498221 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.491069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:39157 (local address 127.4.82.131:34789)
0809 01:04:02.491337 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:02.491345 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:02.492049 (+ 704us) server_negotiation.cc:407] Connection header received
0809 01:04:02.492254 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:02.492262 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:02.492340 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:02.492404 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:02.493262 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.494238 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.496192 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.497210 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.497357 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.497432 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:02.497850 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 01:04:02.498016 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 01:04:02.498766 23195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.490793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:34789 (local address 127.4.82.129:39157)
0809 01:04:02.491893 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:02.491909 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:02.492073 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:02.492637 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:02.492641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:02.492669 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:02.493059 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:02.493070 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.494390 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.494394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:02.496019 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:02.496032 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.497424 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.497431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:02.497622 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.497659 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 01:04:02.498398 (+ 739us) client_negotiation.cc:240] Negotiation successful
0809 01:04:02.498497 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"mutex_wait_us":171,"thread_start_us":512,"threads_started":1}
I20260809 01:04:02.498739 22975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80772a56a3c14c2eb0ff39c9a3abfdc2" candidate_uuid: "a26892ca9161423e8b4eb52c0098f898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be5afc9ceff74e7fbd402fe58cc0953e" is_pre_election: true
I20260809 01:04:02.499329 22975 raft_consensus.cc:2454] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a26892ca9161423e8b4eb52c0098f898 in term 0.
I20260809 01:04:02.499783 23119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80772a56a3c14c2eb0ff39c9a3abfdc2" candidate_uuid: "a26892ca9161423e8b4eb52c0098f898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a4fbd3839f34ff1957c78b4e23d5f98" is_pre_election: true
I20260809 01:04:02.499980 22765 leader_election.cc:302] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a26892ca9161423e8b4eb52c0098f898, be5afc9ceff74e7fbd402fe58cc0953e; no voters:
I20260809 01:04:02.500293 23119 raft_consensus.cc:2454] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a26892ca9161423e8b4eb52c0098f898 in term 0.
I20260809 01:04:02.500411 23191 raft_consensus.cc:2781] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:04:02.500581 23191 raft_consensus.cc:482] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:04:02.500700 23191 raft_consensus.cc:3037] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:02.502848 23191 raft_consensus.cc:504] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be5afc9ceff74e7fbd402fe58cc0953e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 46509 } } peers { permanent_uuid: "4a4fbd3839f34ff1957c78b4e23d5f98" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34789 } } peers { permanent_uuid: "a26892ca9161423e8b4eb52c0098f898" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44177 } }
I20260809 01:04:02.503578 23191 leader_election.cc:288] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [CANDIDATE]: Term 1 election: Requested vote from peers be5afc9ceff74e7fbd402fe58cc0953e (127.4.82.130:46509), 4a4fbd3839f34ff1957c78b4e23d5f98 (127.4.82.131:34789)
I20260809 01:04:02.503968 22975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80772a56a3c14c2eb0ff39c9a3abfdc2" candidate_uuid: "a26892ca9161423e8b4eb52c0098f898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be5afc9ceff74e7fbd402fe58cc0953e"
I20260809 01:04:02.504019 23119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80772a56a3c14c2eb0ff39c9a3abfdc2" candidate_uuid: "a26892ca9161423e8b4eb52c0098f898" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a4fbd3839f34ff1957c78b4e23d5f98"
I20260809 01:04:02.504166 22975 raft_consensus.cc:3037] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:02.504313 23119 raft_consensus.cc:3037] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:02.506364 22975 raft_consensus.cc:2454] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a26892ca9161423e8b4eb52c0098f898 in term 1.
I20260809 01:04:02.506642 23119 raft_consensus.cc:2454] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a26892ca9161423e8b4eb52c0098f898 in term 1.
I20260809 01:04:02.506733 22765 leader_election.cc:302] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a26892ca9161423e8b4eb52c0098f898, be5afc9ceff74e7fbd402fe58cc0953e; no voters:
I20260809 01:04:02.507057 23191 raft_consensus.cc:2781] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:02.507556 23191 raft_consensus.cc:686] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [term 1 LEADER]: Becoming Leader. State: Replica: a26892ca9161423e8b4eb52c0098f898, State: Running, Role: LEADER
I20260809 01:04:02.508052 23191 consensus_queue.cc:237] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be5afc9ceff74e7fbd402fe58cc0953e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 46509 } } peers { permanent_uuid: "4a4fbd3839f34ff1957c78b4e23d5f98" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34789 } } peers { permanent_uuid: "a26892ca9161423e8b4eb52c0098f898" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44177 } }
I20260809 01:04:02.513296 22684 catalog_manager.cc:5495] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 reported cstate change: term changed from 0 to 1, leader changed from <none> to a26892ca9161423e8b4eb52c0098f898 (127.4.82.129). New cstate: current_term: 1 leader_uuid: "a26892ca9161423e8b4eb52c0098f898" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be5afc9ceff74e7fbd402fe58cc0953e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 46509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a4fbd3839f34ff1957c78b4e23d5f98" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a26892ca9161423e8b4eb52c0098f898" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 44177 } health_report { overall_health: HEALTHY } } }
I20260809 01:04:02.562485 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.557336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37248 (local address 127.4.82.129:44177)
0809 01:04:02.557562 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:02.557567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:02.557586 (+ 19us) server_negotiation.cc:407] Connection header received
0809 01:04:02.557682 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:02.557686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:02.557763 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:02.557887 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:04:02.558228 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.558938 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.560322 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.560779 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.560875 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.561345 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:04:02.561911 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:04:02.562025 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:02.562118 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 01:04:02.562185 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 01:04:02.571015 23119 raft_consensus.cc:1261] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98 [term 1 FOLLOWER]: Refusing update from remote peer a26892ca9161423e8b4eb52c0098f898: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:04:02.571015 22975 raft_consensus.cc:1261] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e [term 1 FOLLOWER]: Refusing update from remote peer a26892ca9161423e8b4eb52c0098f898: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:04:02.572106 23196 consensus_queue.cc:1035] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a4fbd3839f34ff1957c78b4e23d5f98" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:04:02.572851 23196 consensus_queue.cc:1035] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be5afc9ceff74e7fbd402fe58cc0953e" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 46509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:04:02.602126 22685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:04:02.602453 22685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:04:02.606261 22685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad472a1779964d218579c51368deee48: Sending DeleteTablet for 3 replicas of tablet 80772a56a3c14c2eb0ff39c9a3abfdc2
I20260809 01:04:02.607398 22952 tablet_service.cc:1514] Processing DeleteTablet for tablet 80772a56a3c14c2eb0ff39c9a3abfdc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:02 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:60682
I20260809 01:04:02.607506 23098 tablet_service.cc:1514] Processing DeleteTablet for tablet 80772a56a3c14c2eb0ff39c9a3abfdc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:02 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:42858
I20260809 01:04:02.608109 22815 tablet_service.cc:1514] Processing DeleteTablet for tablet 80772a56a3c14c2eb0ff39c9a3abfdc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:02 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:37236
I20260809 01:04:02.609690 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 22748
I20260809 01:04:02.619871 23211 tablet_replica.cc:330] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898: stopping tablet replica
I20260809 01:04:02.620381 23211 raft_consensus.cc:2229] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:04:02.621352 23211 raft_consensus.cc:2258] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:02.624112 23211 ts_tablet_manager.cc:1905] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:02.633289 23211 ts_tablet_manager.cc:1918] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:02.633551 23211 log.cc:1196] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/80772a56a3c14c2eb0ff39c9a3abfdc2
I20260809 01:04:02.634271 23211 ts_tablet_manager.cc:1939] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P a26892ca9161423e8b4eb52c0098f898: Deleting consensus metadata
I20260809 01:04:02.636282 22667 catalog_manager.cc:4841] TS a26892ca9161423e8b4eb52c0098f898 (127.4.82.129:44177): tablet 80772a56a3c14c2eb0ff39c9a3abfdc2 (table test-table [id=63db594ea0874d818b680d85774bc90b]) successfully deleted
I20260809 01:04:02.638207 23209 tablet_replica.cc:330] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e: stopping tablet replica
I20260809 01:04:02.638386 23210 tablet_replica.cc:330] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98: stopping tablet replica
I20260809 01:04:02.638873 23210 raft_consensus.cc:2229] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:04:02.638866 23209 raft_consensus.cc:2229] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:04:02.639472 23210 raft_consensus.cc:2258] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:02.639488 23209 raft_consensus.cc:2258] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:02.641777 23210 ts_tablet_manager.cc:1905] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:02.641866 23209 ts_tablet_manager.cc:1905] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 01:04:02 dist-test-slave-07c2 krb5kdc[22637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 01:04:02.649297 23209 ts_tablet_manager.cc:1918] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:02.649554 23209 log.cc:1196] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/80772a56a3c14c2eb0ff39c9a3abfdc2
I20260809 01:04:02.649621 23210 ts_tablet_manager.cc:1918] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:02.649839 23210 log.cc:1196] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/80772a56a3c14c2eb0ff39c9a3abfdc2
I20260809 01:04:02.650177 23209 ts_tablet_manager.cc:1939] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P be5afc9ceff74e7fbd402fe58cc0953e: Deleting consensus metadata
I20260809 01:04:02.650782 23210 ts_tablet_manager.cc:1939] T 80772a56a3c14c2eb0ff39c9a3abfdc2 P 4a4fbd3839f34ff1957c78b4e23d5f98: Deleting consensus metadata
I20260809 01:04:02.652228 22668 catalog_manager.cc:4841] TS be5afc9ceff74e7fbd402fe58cc0953e (127.4.82.130:46509): tablet 80772a56a3c14c2eb0ff39c9a3abfdc2 (table test-table [id=63db594ea0874d818b680d85774bc90b]) successfully deleted
I20260809 01:04:02.652802 22670 catalog_manager.cc:4841] TS 4a4fbd3839f34ff1957c78b4e23d5f98 (127.4.82.131:34789): tablet 80772a56a3c14c2eb0ff39c9a3abfdc2 (table test-table [id=63db594ea0874d818b680d85774bc90b]) successfully deleted
I20260809 01:04:02.656145 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.611425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.4.82.129:44177)
0809 01:04:02.611650 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:02.611659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:02.635947 (+ 24288us) server_negotiation.cc:407] Connection header received
0809 01:04:02.636968 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:02.636978 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:02.637082 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:02.637198 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:02.639233 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.640489 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.642456 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.643195 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.643315 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.649158 (+ 5843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:02.649175 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:02.649193 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:02.649247 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:02.653147 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:02.654006 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:02.654012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:02.654014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:02.654159 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:02.654593 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:02.654596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:02.654598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:02.655021 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:02.655227 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:02.655647 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 01:04:02.655844 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 01:04:02.656864 22876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:02.769214 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 22748
I20260809 01:04:02.789872 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 22888
Aug 09 01:04:02 dist-test-slave-07c2 krb5kdc[22637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 01:04:02.805029 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.790617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60688 (local address 127.4.82.130:46509)
0809 01:04:02.790816 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:02.790824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:02.790855 (+ 31us) server_negotiation.cc:407] Connection header received
0809 01:04:02.791078 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:02.791082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:02.791160 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:02.791273 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:02.792566 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.793305 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.794750 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.795134 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.795272 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.799524 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:02.799538 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:02.799553 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:02.799585 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:02.802155 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:02.802855 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:02.802863 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:02.802865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:02.802984 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:02.803390 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:02.803397 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:02.803400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:02.803799 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:02.804113 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:02.804492 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 01:04:02.804655 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 01:04:02.805713 23019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:02.894277 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 22888
I20260809 01:04:02.913589 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 23030
Aug 09 01:04:02 dist-test-slave-07c2 krb5kdc[22637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 01:04:02.928179 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:02.914281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42872 (local address 127.4.82.131:34789)
0809 01:04:02.914478 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:02.914482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:02.914500 (+ 18us) server_negotiation.cc:407] Connection header received
0809 01:04:02.914614 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:02.914619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:02.914685 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:02.914786 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:02.916109 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.916849 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.918141 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:02.918511 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:02.918598 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:02.922917 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:02.922930 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:02.922946 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:02.922978 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:02.925766 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:02.926385 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:02.926392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:02.926395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:02.926531 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:02.926845 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:02.926848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:02.926850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:02.927164 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:02.927442 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:02.927643 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 01:04:02.927813 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 01:04:02.928793 23161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:03.017194 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 23030
I20260809 01:04:03.035683 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 22650
I20260809 01:04:03.036859 22716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:03.147033 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 22650
2026-08-09T01:04:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4551 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T01:04:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:03Z Disabled control of system clock
I20260809 01:04:03.236842 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:42779
--webserver_interface=127.4.82.190
--webserver_port=0
--builtin_ntp_servers=127.4.82.148:42025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:42779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 01:04:03.575372 23220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:03.575758 23220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:03.575938 23220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:03.584529 23220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:03.584640 23220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:03.584729 23220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:03.584829 23220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:03.584909 23220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:03.594703 23220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:42779
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:42779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:03.596096 23220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:03.597947 23220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:03.610658 23225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:03.610692 23228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:03.610754 23226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:03.611934 23220 server_base.cc:1034] running on GCE node
I20260809 01:04:03.612749 23220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:03.614169 23220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:03.615442 23220 hybrid_clock.cc:648] HybridClock initialized: now 1786237443615397 us; error 37 us; skew 500 ppm
I20260809 01:04:03.615859 23220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:04:03.622881 23220 webserver.cc:460] Webserver started at http://127.4.82.190:39847/ using document root <none> and password file <none>
I20260809 01:04:03.623703 23220 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:03.623821 23220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:03.624158 23220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:03.627099 23220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "aeb0dadb653b4f9e8afd94319a125025"
format_stamp: "Formatted at 2026-08-09 01:04:03 on dist-test-slave-07c2"
server_key: "40057c58bbd918e07551a0f6af7d40f2"
server_key_iv: "d06c6220bf713fbc11e807a6cf463579"
server_key_version: "encryptionkey@0"
I20260809 01:04:03.627991 23220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "aeb0dadb653b4f9e8afd94319a125025"
format_stamp: "Formatted at 2026-08-09 01:04:03 on dist-test-slave-07c2"
server_key: "40057c58bbd918e07551a0f6af7d40f2"
server_key_iv: "d06c6220bf713fbc11e807a6cf463579"
server_key_version: "encryptionkey@0"
I20260809 01:04:03.633090 23220 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 01:04:03.636543 23235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:03.637916 23220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 01:04:03.638139 23220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "aeb0dadb653b4f9e8afd94319a125025"
format_stamp: "Formatted at 2026-08-09 01:04:03 on dist-test-slave-07c2"
server_key: "40057c58bbd918e07551a0f6af7d40f2"
server_key_iv: "d06c6220bf713fbc11e807a6cf463579"
server_key_version: "encryptionkey@0"
I20260809 01:04:03.638366 23220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:03.660874 23220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:03.661760 23220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:03.662071 23220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:03.681340 23220 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:42779
I20260809 01:04:03.681360 23288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:42779 every 8 connection(s)
I20260809 01:04:03.682729 23220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:03.683521 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 23220
I20260809 01:04:03.683718 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:03.684361 4426 external_mini_cluster.cc:1428] Setting key 6a2f567291f332ca5f7b8adc85576ad8
I20260809 01:04:03.687171 23289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:03.697569 23292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:03.686260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.4.82.190:42779)
0809 01:04:03.687283 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:03.687320 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:03.687371 (+ 51us) server_negotiation.cc:407] Connection header received
0809 01:04:03.689005 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:03.689024 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:03.689427 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:03.689839 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:03.690391 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:03.691798 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:03.693272 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:03.693916 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:03.694111 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:03.695402 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:03.695411 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:03.695423 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:03.695977 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:03.696051 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:03.696071 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:03.696198 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:03.696419 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 01:04:03.696710 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":380,"threads_started":1}
I20260809 01:04:03.697664 23289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025: Bootstrap starting.
I20260809 01:04:03.701260 23289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:03.702273 23289 log.cc:824] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:03.705075 23289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025: No bootstrap required, opened a new log
I20260809 01:04:03.712512 23289 raft_consensus.cc:348] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb0dadb653b4f9e8afd94319a125025" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 42779 } }
I20260809 01:04:03.712846 23289 raft_consensus.cc:374] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:03.712963 23289 raft_consensus.cc:729] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeb0dadb653b4f9e8afd94319a125025, State: Initialized, Role: FOLLOWER
I20260809 01:04:03.713500 23289 consensus_queue.cc:260] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb0dadb653b4f9e8afd94319a125025" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 42779 } }
I20260809 01:04:03.713722 23289 raft_consensus.cc:388] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:03.713858 23289 raft_consensus.cc:482] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:03.714032 23289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:03.716174 23289 raft_consensus.cc:504] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb0dadb653b4f9e8afd94319a125025" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 42779 } }
I20260809 01:04:03.716648 23289 leader_election.cc:302] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aeb0dadb653b4f9e8afd94319a125025; no voters:
I20260809 01:04:03.717227 23289 leader_election.cc:288] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:03.717401 23294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:03.718122 23294 raft_consensus.cc:686] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [term 1 LEADER]: Becoming Leader. State: Replica: aeb0dadb653b4f9e8afd94319a125025, State: Running, Role: LEADER
I20260809 01:04:03.718763 23294 consensus_queue.cc:237] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb0dadb653b4f9e8afd94319a125025" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 42779 } }
I20260809 01:04:03.719447 23289 sys_catalog.cc:564] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:03.721736 23295 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aeb0dadb653b4f9e8afd94319a125025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb0dadb653b4f9e8afd94319a125025" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 42779 } } }
I20260809 01:04:03.722191 23295 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:03.721700 23297 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aeb0dadb653b4f9e8afd94319a125025. Latest consensus state: current_term: 1 leader_uuid: "aeb0dadb653b4f9e8afd94319a125025" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeb0dadb653b4f9e8afd94319a125025" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 42779 } } }
I20260809 01:04:03.722366 23297 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:03.724537 23302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:03.730381 23302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:03.743340 23302 catalog_manager.cc:1347] Generated new cluster ID: f85fc99d03204bc0812ec5e475a56f8c
I20260809 01:04:03.743458 23302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:03.756975 23302 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:03.758172 23302 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:03.765708 23302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025: Generated new TSK 0
I20260809 01:04:03.766465 23302 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:03.776993 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:42779
--builtin_ntp_servers=127.4.82.148:42025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 01:04:04.110611 23314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:04.110885 23314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:04.110971 23314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:04.118008 23314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:04.118095 23314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:04.118203 23314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:04:04.127234 23314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:42779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:04.128407 23314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:04.130095 23314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:04.142418 23320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:04.142424 23323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:04.142714 23319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:04.143517 23314 server_base.cc:1034] running on GCE node
I20260809 01:04:04.144058 23314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:04.144914 23314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:04.146167 23314 hybrid_clock.cc:648] HybridClock initialized: now 1786237444146111 us; error 40 us; skew 500 ppm
I20260809 01:04:04.146490 23314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:04:04.148681 23314 webserver.cc:460] Webserver started at http://127.4.82.129:35773/ using document root <none> and password file <none>
I20260809 01:04:04.149323 23314 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:04.149442 23314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:04.149756 23314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:04.152535 23314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "e9ef985d01c54d01865af07a542c065b"
format_stamp: "Formatted at 2026-08-09 01:04:04 on dist-test-slave-07c2"
server_key: "d92ea91ea47411fd8846ff04978021d7"
server_key_iv: "97faa37217955727a5cef940708efca8"
server_key_version: "encryptionkey@0"
I20260809 01:04:04.153312 23314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "e9ef985d01c54d01865af07a542c065b"
format_stamp: "Formatted at 2026-08-09 01:04:04 on dist-test-slave-07c2"
server_key: "d92ea91ea47411fd8846ff04978021d7"
server_key_iv: "97faa37217955727a5cef940708efca8"
server_key_version: "encryptionkey@0"
I20260809 01:04:04.158519 23314 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 01:04:04.161871 23329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:04.163087 23314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 01:04:04.163264 23314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "e9ef985d01c54d01865af07a542c065b"
format_stamp: "Formatted at 2026-08-09 01:04:04 on dist-test-slave-07c2"
server_key: "d92ea91ea47411fd8846ff04978021d7"
server_key_iv: "97faa37217955727a5cef940708efca8"
server_key_version: "encryptionkey@0"
I20260809 01:04:04.163528 23314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:04.196617 23314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:04.197427 23314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:04.197742 23314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:04.198741 23314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:04.200150 23314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:04.200279 23314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:04.200412 23314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:04.200500 23314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:04.233287 23314 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:45867
I20260809 01:04:04.233374 23447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:45867 every 8 connection(s)
I20260809 01:04:04.234546 23314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:04:04.235503 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 23314
I20260809 01:04:04.235693 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:04:04.236189 4426 external_mini_cluster.cc:1428] Setting key f30483348e5e3bd7a26cd52ebdaa0bfd
I20260809 01:04:04.239410 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:42779
--builtin_ntp_servers=127.4.82.148:42025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 01:04:04.247423 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:04.237249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:50695 (local address 127.4.82.190:42779)
0809 01:04:04.238340 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:04.238345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:04.239159 (+ 814us) server_negotiation.cc:407] Connection header received
0809 01:04:04.241044 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:04.241049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:04.241139 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:04.241263 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:04.242316 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:04.243046 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:04.244627 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:04.245033 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:04.245171 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:04.246524 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:04.246529 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:04.246532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:04.246550 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:04.246594 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:04.246605 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:04.246718 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:04.246951 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 01:04:04.247045 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":951,"thread_start_us":844,"threads_started":1}
I20260809 01:04:04.248346 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:04.238029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:42779 (local address 127.4.82.129:50695)
0809 01:04:04.239012 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:04.239047 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:04.240787 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:04.241490 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:04.241498 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:04.241578 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:04.242150 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:04.242166 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:04.243210 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:04.243215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:04.244471 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:04.244488 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:04.245257 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:04.245267 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:04.245509 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:04.246234 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:04:04.246237 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:04.246272 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:04:04.246337 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:04.246731 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:04.246744 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:04.246753 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 01:04:04.246935 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 01:04:04.247298 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":338,"threads_started":1}
I20260809 01:04:04.249892 23448 heartbeater.cc:347] Connected to a master server at 127.4.82.190:42779
I20260809 01:04:04.250175 23448 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:04.250924 23448 heartbeater.cc:511] Master 127.4.82.190:42779 requested a full tablet report, sending...
I20260809 01:04:04.253093 23252 ts_manager.cc:194] Registered new tserver with Master: e9ef985d01c54d01865af07a542c065b (127.4.82.129:45867)
I20260809 01:04:04.254598 23252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.82.129:50695
W20260809 01:04:04.568324 23452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:04.568595 23452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:04.568770 23452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:04.575841 23452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:04.575927 23452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:04.576134 23452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:04:04.584851 23452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:42779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:04.586058 23452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:04.587857 23452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:04.600428 23457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:04.600382 23458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:04.600468 23460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:04.601274 23452 server_base.cc:1034] running on GCE node
I20260809 01:04:04.601807 23452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:04.602676 23452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:04.603896 23452 hybrid_clock.cc:648] HybridClock initialized: now 1786237444603855 us; error 54 us; skew 500 ppm
I20260809 01:04:04.604190 23452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:04:04.606279 23452 webserver.cc:460] Webserver started at http://127.4.82.130:34163/ using document root <none> and password file <none>
I20260809 01:04:04.607065 23452 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:04.607206 23452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:04.607551 23452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:04.610301 23452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "f63946e9c17a4e2fa204f15c6e66aa75"
format_stamp: "Formatted at 2026-08-09 01:04:04 on dist-test-slave-07c2"
server_key: "0ea1bc46b65dd2b17012dea5e401f184"
server_key_iv: "f3e6082243dda23793fc28453f80ae5f"
server_key_version: "encryptionkey@0"
I20260809 01:04:04.611001 23452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "f63946e9c17a4e2fa204f15c6e66aa75"
format_stamp: "Formatted at 2026-08-09 01:04:04 on dist-test-slave-07c2"
server_key: "0ea1bc46b65dd2b17012dea5e401f184"
server_key_iv: "f3e6082243dda23793fc28453f80ae5f"
server_key_version: "encryptionkey@0"
I20260809 01:04:04.615935 23452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 01:04:04.619170 23466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:04.620414 23452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 01:04:04.620565 23452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "f63946e9c17a4e2fa204f15c6e66aa75"
format_stamp: "Formatted at 2026-08-09 01:04:04 on dist-test-slave-07c2"
server_key: "0ea1bc46b65dd2b17012dea5e401f184"
server_key_iv: "f3e6082243dda23793fc28453f80ae5f"
server_key_version: "encryptionkey@0"
I20260809 01:04:04.620759 23452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:04.639835 23452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:04.640534 23452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:04.640820 23452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:04.641733 23452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:04.643246 23452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:04.643391 23452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 01:04:04.643513 23452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:04.643603 23452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:04.676014 23452 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:40335
I20260809 01:04:04.676100 23583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:40335 every 8 connection(s)
I20260809 01:04:04.677287 23452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:04:04.678588 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 23452
I20260809 01:04:04.678798 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:04:04.679451 4426 external_mini_cluster.cc:1428] Setting key 248b966c9c77f89b5a38f48fce2bdbae
I20260809 01:04:04.682772 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:42779
--builtin_ntp_servers=127.4.82.148:42025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 01:04:04.693094 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:04.680033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:46791 (local address 127.4.82.190:42779)
0809 01:04:04.680211 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:04.680216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:04.681906 (+ 1690us) server_negotiation.cc:407] Connection header received
0809 01:04:04.684683 (+ 2777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:04.684689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:04.684766 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:04.684890 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:04.686188 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:04.687376 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:04.689802 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:04.690439 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:04.690553 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:04.692251 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:04.692258 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:04.692262 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:04.692286 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:04.692327 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:04.692346 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:04.692448 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:04.692690 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 01:04:04.692785 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 01:04:04.694334 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:04.680644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:42779 (local address 127.4.82.130:46791)
0809 01:04:04.681759 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:04.681808 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:04.684391 (+ 2583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:04.685106 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:04.685119 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:04.685226 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:04.686002 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:04.686028 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:04.687532 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:04.687540 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:04.689622 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:04.689640 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:04.690675 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:04.690685 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:04.691029 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:04.691862 (+ 833us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:04:04.691868 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:04.691918 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:04:04.692016 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:04.692466 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:04.692479 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:04.692489 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 01:04:04.692674 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 01:04:04.693054 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":349,"threads_started":1}
I20260809 01:04:04.695674 23584 heartbeater.cc:347] Connected to a master server at 127.4.82.190:42779
I20260809 01:04:04.695986 23584 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:04.696657 23584 heartbeater.cc:511] Master 127.4.82.190:42779 requested a full tablet report, sending...
I20260809 01:04:04.698011 23252 ts_manager.cc:194] Registered new tserver with Master: f63946e9c17a4e2fa204f15c6e66aa75 (127.4.82.130:40335)
I20260809 01:04:04.698808 23252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.82.130:46791
W20260809 01:04:05.022894 23587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:05.023170 23587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:05.023360 23587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:05.030426 23587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:05.030509 23587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:05.030634 23587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:04:05.039417 23587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:42025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:42779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:05.040598 23587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:05.042356 23587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:05.054998 23592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:05.055053 23593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:05.055107 23595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:05.056034 23587 server_base.cc:1034] running on GCE node
I20260809 01:04:05.056584 23587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:05.057452 23587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:05.058677 23587 hybrid_clock.cc:648] HybridClock initialized: now 1786237445058633 us; error 55 us; skew 500 ppm
I20260809 01:04:05.058990 23587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:04:05.061210 23587 webserver.cc:460] Webserver started at http://127.4.82.131:39905/ using document root <none> and password file <none>
I20260809 01:04:05.061863 23587 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:05.061998 23587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:05.062314 23587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:05.065065 23587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "bf3fb3a72e804928a2905e11e400bd77"
format_stamp: "Formatted at 2026-08-09 01:04:05 on dist-test-slave-07c2"
server_key: "e68bb4dbfcfe1cf3ffe8b351ac40c540"
server_key_iv: "e9b1dc8d24a5647a916fcc8f0c7e16da"
server_key_version: "encryptionkey@0"
I20260809 01:04:05.065838 23587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "bf3fb3a72e804928a2905e11e400bd77"
format_stamp: "Formatted at 2026-08-09 01:04:05 on dist-test-slave-07c2"
server_key: "e68bb4dbfcfe1cf3ffe8b351ac40c540"
server_key_iv: "e9b1dc8d24a5647a916fcc8f0c7e16da"
server_key_version: "encryptionkey@0"
I20260809 01:04:05.070756 23587 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 01:04:05.074031 23602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:05.075258 23587 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 01:04:05.075484 23587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "bf3fb3a72e804928a2905e11e400bd77"
format_stamp: "Formatted at 2026-08-09 01:04:05 on dist-test-slave-07c2"
server_key: "e68bb4dbfcfe1cf3ffe8b351ac40c540"
server_key_iv: "e9b1dc8d24a5647a916fcc8f0c7e16da"
server_key_version: "encryptionkey@0"
I20260809 01:04:05.075697 23587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:05.097340 23587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:05.098114 23587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:05.098428 23587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:05.099380 23587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:05.100834 23587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:05.100962 23587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:05.101096 23587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:05.101190 23587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:05.133965 23587 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:41207
I20260809 01:04:05.134054 23719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:41207 every 8 connection(s)
I20260809 01:04:05.135423 23587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:04:05.145524 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 23587
I20260809 01:04:05.145736 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:04:05.146200 4426 external_mini_cluster.cc:1428] Setting key cca19ef1d6d436d9d5c2997b866aef6a
I20260809 01:04:05.147838 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.138017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:38449 (local address 127.4.82.190:42779)
0809 01:04:05.138209 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:05.138220 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:05.139695 (+ 1475us) server_negotiation.cc:407] Connection header received
0809 01:04:05.141482 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:05.141489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:05.141584 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:05.141695 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:05.142698 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.143520 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.144993 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.145430 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.145567 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.146972 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:05.146983 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:05.146986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:05.147004 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:05.147035 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:05.147046 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:05.147150 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:05.147408 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 01:04:05.147506 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 01:04:05.148752 23722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.138523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:42779 (local address 127.4.82.131:38449)
0809 01:04:05.139533 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.139579 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:05.141218 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:05.141927 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:05.141935 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:05.142023 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:05.142530 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:05.142545 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.143666 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.143672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:05.144853 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:05.144864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.145651 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.145657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:05.145945 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.146659 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:04:05.146662 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:05.146695 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:04:05.146773 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:05.147148 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:05.147160 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:05.147171 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 01:04:05.147393 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 01:04:05.147742 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":286,"threads_started":1}
I20260809 01:04:05.150029 23720 heartbeater.cc:347] Connected to a master server at 127.4.82.190:42779
I20260809 01:04:05.150310 23720 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:05.150980 23720 heartbeater.cc:511] Master 127.4.82.190:42779 requested a full tablet report, sending...
I20260809 01:04:05.152280 23252 ts_manager.cc:194] Registered new tserver with Master: bf3fb3a72e804928a2905e11e400bd77 (127.4.82.131:41207)
I20260809 01:04:05.153064 23252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.82.131:38449
I20260809 01:04:05.161219 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:04:05.171205 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.165239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50716 (local address 127.4.82.190:42779)
0809 01:04:05.165453 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:05.165462 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:05.165855 (+ 393us) server_negotiation.cc:407] Connection header received
0809 01:04:05.166060 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:05.166064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:05.166137 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:05.166242 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:05.166749 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.167576 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.168974 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.169379 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.169516 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.170517 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:05.170521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:05.170524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:05.170540 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:05.170569 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:05.170580 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:05.170692 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:05.170830 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 01:04:05.170915 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 01:04:05.174584 23252 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:04:05.181485 23252 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:04:05.203418 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.192701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:45867 (local address 127.0.0.1:54280)
0809 01:04:05.193402 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.193456 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:05.193635 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:05.195932 (+ 2297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:05.195938 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:05.195966 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:05.196274 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:05.196285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.198209 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.198214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:05.200516 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:05.200531 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.202418 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.202424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:05.202634 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.202695 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 01:04:05.202856 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 01:04:05.203042 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":259,"threads_started":1}
I20260809 01:04:05.203428 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.192907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.4.82.129:45867)
0809 01:04:05.195225 (+ 2318us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:05.195242 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:05.195274 (+ 32us) server_negotiation.cc:407] Connection header received
0809 01:04:05.195392 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:05.195399 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:05.195639 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:05.195919 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:05.196432 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.198057 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.200750 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.202221 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.202378 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.202622 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:05.202893 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 01:04:05.203054 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2138,"thread_start_us":2345,"threads_started":1}
I20260809 01:04:05.203671 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.193844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48174 (local address 127.4.82.130:40335)
0809 01:04:05.194384 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:05.194391 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:05.194927 (+ 536us) server_negotiation.cc:407] Connection header received
0809 01:04:05.195837 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:05.195848 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:05.196092 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:05.196270 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:05.197096 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.198563 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.200694 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.202210 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.202349 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.202582 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:05.203173 (+ 591us) server_negotiation.cc:299] Negotiation successful
0809 01:04:05.203354 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":904,"threads_started":1}
I20260809 01:04:05.204406 23737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.193208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:40335 (local address 127.0.0.1:48174)
0809 01:04:05.194822 (+ 1614us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.194840 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:05.195048 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:05.196501 (+ 1453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:05.196505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:05.196531 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:05.196882 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:05.196894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.198723 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.198738 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:05.200517 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:05.200528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.202733 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.202740 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:05.202962 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.202995 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 01:04:05.203925 (+ 930us) client_negotiation.cc:240] Negotiation successful
0809 01:04:05.204056 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1461,"mutex_wait_us":60,"thread_start_us":4008,"threads_started":1}
I20260809 01:04:05.206234 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.193915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:41207 (local address 127.0.0.1:52950)
0809 01:04:05.197441 (+ 3526us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.197454 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:05.197587 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:05.198655 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:05.198659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:05.198685 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:05.199264 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:05.199303 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.201137 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.201145 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:05.203487 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:05.203502 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.205496 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.205503 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:05.205675 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.205705 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 01:04:05.205785 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 01:04:05.205899 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3416,"mutex_wait_us":2,"thread_start_us":595,"threads_started":1}
I20260809 01:04:05.206357 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.194419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52950 (local address 127.4.82.131:41207)
0809 01:04:05.197022 (+ 2603us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:05.197033 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:05.197548 (+ 515us) server_negotiation.cc:407] Connection header received
0809 01:04:05.197971 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:05.197978 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:05.198194 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:05.198366 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:05.199498 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.200988 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.203941 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.205308 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.205443 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.205700 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:05.205840 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 01:04:05.205985 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2416,"thread_start_us":2265,"threads_started":1}
I20260809 01:04:05.207276 23378 tablet_service.cc:1467] Processing CreateTablet for tablet bed00d37928d493f83899eb484ae1ae8 (DEFAULT_TABLE table=test-table [id=341e4721acfd43718f5c544c03b454e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:05.207767 23517 tablet_service.cc:1467] Processing CreateTablet for tablet bed00d37928d493f83899eb484ae1ae8 (DEFAULT_TABLE table=test-table [id=341e4721acfd43718f5c544c03b454e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:05.209110 23378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bed00d37928d493f83899eb484ae1ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:05.209194 23517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bed00d37928d493f83899eb484ae1ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:05.209668 23651 tablet_service.cc:1467] Processing CreateTablet for tablet bed00d37928d493f83899eb484ae1ae8 (DEFAULT_TABLE table=test-table [id=341e4721acfd43718f5c544c03b454e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:05.210826 23651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bed00d37928d493f83899eb484ae1ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:05.221868 23739 tablet_bootstrap.cc:492] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77: Bootstrap starting.
I20260809 01:04:05.224756 23741 tablet_bootstrap.cc:492] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75: Bootstrap starting.
I20260809 01:04:05.224041 23740 tablet_bootstrap.cc:492] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b: Bootstrap starting.
I20260809 01:04:05.226131 23739 tablet_bootstrap.cc:654] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:05.227442 23739 log.cc:824] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:05.228718 23740 tablet_bootstrap.cc:654] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:05.230022 23741 tablet_bootstrap.cc:654] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:05.229996 23740 log.cc:824] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:05.230849 23739 tablet_bootstrap.cc:492] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77: No bootstrap required, opened a new log
I20260809 01:04:05.231119 23739 ts_tablet_manager.cc:1397] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260809 01:04:05.231158 23741 log.cc:824] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:05.233117 23740 tablet_bootstrap.cc:492] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b: No bootstrap required, opened a new log
I20260809 01:04:05.233544 23740 ts_tablet_manager.cc:1397] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 01:04:05.233892 23741 tablet_bootstrap.cc:492] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75: No bootstrap required, opened a new log
I20260809 01:04:05.234176 23741 ts_tablet_manager.cc:1397] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 01:04:05.241492 23739 raft_consensus.cc:348] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef985d01c54d01865af07a542c065b" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45867 } } peers { permanent_uuid: "bf3fb3a72e804928a2905e11e400bd77" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 41207 } } peers { permanent_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 40335 } }
I20260809 01:04:05.242028 23739 raft_consensus.cc:374] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:05.242165 23739 raft_consensus.cc:729] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf3fb3a72e804928a2905e11e400bd77, State: Initialized, Role: FOLLOWER
I20260809 01:04:05.242936 23739 consensus_queue.cc:260] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef985d01c54d01865af07a542c065b" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45867 } } peers { permanent_uuid: "bf3fb3a72e804928a2905e11e400bd77" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 41207 } } peers { permanent_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 40335 } }
I20260809 01:04:05.243191 23741 raft_consensus.cc:348] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef985d01c54d01865af07a542c065b" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45867 } } peers { permanent_uuid: "bf3fb3a72e804928a2905e11e400bd77" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 41207 } } peers { permanent_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 40335 } }
I20260809 01:04:05.243642 23741 raft_consensus.cc:374] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:05.243760 23741 raft_consensus.cc:729] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f63946e9c17a4e2fa204f15c6e66aa75, State: Initialized, Role: FOLLOWER
I20260809 01:04:05.244370 23741 consensus_queue.cc:260] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef985d01c54d01865af07a542c065b" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45867 } } peers { permanent_uuid: "bf3fb3a72e804928a2905e11e400bd77" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 41207 } } peers { permanent_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 40335 } }
I20260809 01:04:05.246556 23740 raft_consensus.cc:348] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef985d01c54d01865af07a542c065b" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45867 } } peers { permanent_uuid: "bf3fb3a72e804928a2905e11e400bd77" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 41207 } } peers { permanent_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 40335 } }
I20260809 01:04:05.247141 23740 raft_consensus.cc:374] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:05.247363 23740 raft_consensus.cc:729] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9ef985d01c54d01865af07a542c065b, State: Initialized, Role: FOLLOWER
I20260809 01:04:05.248226 23740 consensus_queue.cc:260] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef985d01c54d01865af07a542c065b" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45867 } } peers { permanent_uuid: "bf3fb3a72e804928a2905e11e400bd77" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 41207 } } peers { permanent_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 40335 } }
I20260809 01:04:05.251540 23720 heartbeater.cc:503] Master 127.4.82.190:42779 was elected leader, sending a full tablet report...
I20260809 01:04:05.252390 23741 ts_tablet_manager.cc:1428] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75: Time spent starting tablet: real 0.018s user 0.012s sys 0.005s
I20260809 01:04:05.252691 23584 heartbeater.cc:503] Master 127.4.82.190:42779 was elected leader, sending a full tablet report...
I20260809 01:04:05.254202 23448 heartbeater.cc:503] Master 127.4.82.190:42779 was elected leader, sending a full tablet report...
I20260809 01:04:05.255098 23740 ts_tablet_manager.cc:1428] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b: Time spent starting tablet: real 0.021s user 0.012s sys 0.005s
I20260809 01:04:05.257100 23739 ts_tablet_manager.cc:1428] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77: Time spent starting tablet: real 0.026s user 0.018s sys 0.005s
I20260809 01:04:05.272909 23747 raft_consensus.cc:482] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:04:05.273232 23747 raft_consensus.cc:504] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef985d01c54d01865af07a542c065b" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45867 } } peers { permanent_uuid: "bf3fb3a72e804928a2905e11e400bd77" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 41207 } } peers { permanent_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 40335 } }
I20260809 01:04:05.274958 23747 leader_election.cc:288] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9ef985d01c54d01865af07a542c065b (127.4.82.129:45867), bf3fb3a72e804928a2905e11e400bd77 (127.4.82.131:41207)
I20260809 01:04:05.282655 23749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.275380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:45867 (local address 127.4.82.130:46725)
0809 01:04:05.276182 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.276209 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:05.276429 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:05.277092 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:05.277096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:05.277121 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:05.277614 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:05.277638 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.278802 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.278807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:05.280614 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:05.280632 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.281893 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.281899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:05.282083 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.282117 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 01:04:05.282209 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 01:04:05.282301 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":445,"threads_started":1}
I20260809 01:04:05.282696 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.275428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:46725 (local address 127.4.82.129:45867)
0809 01:04:05.275622 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:05.275627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:05.276299 (+ 672us) server_negotiation.cc:407] Connection header received
0809 01:04:05.276610 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:05.276616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:05.276706 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:05.276781 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:05.277815 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.278667 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.280815 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.281675 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.281810 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.281860 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:05.282287 (+ 427us) server_negotiation.cc:299] Negotiation successful
0809 01:04:05.282429 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 01:04:05.282757 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.276429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:43803 (local address 127.4.82.131:41207)
0809 01:04:05.276648 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:05.276654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:05.277209 (+ 555us) server_negotiation.cc:407] Connection header received
0809 01:04:05.277370 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:05.277376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:05.277456 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:05.277566 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:05.278154 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.279061 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.281053 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.281887 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.282044 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.282109 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:05.282397 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 01:04:05.282519 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 01:04:05.283432 23750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.276262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:41207 (local address 127.4.82.130:43803)
0809 01:04:05.277090 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.277104 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:05.277227 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:05.277584 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:05.277590 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:05.277608 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:05.277997 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:05.278008 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.279190 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.279194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:05.280864 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:05.280879 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.282062 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.282067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:05.282223 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.282260 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 01:04:05.282352 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 01:04:05.282468 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":1,"thread_start_us":539,"threads_started":1}
I20260809 01:04:05.284173 23399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bed00d37928d493f83899eb484ae1ae8" candidate_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9ef985d01c54d01865af07a542c065b" is_pre_election: true
I20260809 01:04:05.284713 23399 raft_consensus.cc:2454] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f63946e9c17a4e2fa204f15c6e66aa75 in term 0.
I20260809 01:04:05.284603 23673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bed00d37928d493f83899eb484ae1ae8" candidate_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf3fb3a72e804928a2905e11e400bd77" is_pre_election: true
I20260809 01:04:05.285063 23673 raft_consensus.cc:2454] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f63946e9c17a4e2fa204f15c6e66aa75 in term 0.
I20260809 01:04:05.285400 23469 leader_election.cc:302] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e9ef985d01c54d01865af07a542c065b, f63946e9c17a4e2fa204f15c6e66aa75; no voters:
I20260809 01:04:05.285846 23747 raft_consensus.cc:2781] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:04:05.285997 23747 raft_consensus.cc:482] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:04:05.286113 23747 raft_consensus.cc:3037] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:05.288403 23747 raft_consensus.cc:504] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef985d01c54d01865af07a542c065b" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45867 } } peers { permanent_uuid: "bf3fb3a72e804928a2905e11e400bd77" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 41207 } } peers { permanent_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 40335 } }
I20260809 01:04:05.288990 23747 leader_election.cc:288] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [CANDIDATE]: Term 1 election: Requested vote from peers e9ef985d01c54d01865af07a542c065b (127.4.82.129:45867), bf3fb3a72e804928a2905e11e400bd77 (127.4.82.131:41207)
I20260809 01:04:05.289409 23399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bed00d37928d493f83899eb484ae1ae8" candidate_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9ef985d01c54d01865af07a542c065b"
I20260809 01:04:05.289486 23673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bed00d37928d493f83899eb484ae1ae8" candidate_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf3fb3a72e804928a2905e11e400bd77"
I20260809 01:04:05.289613 23399 raft_consensus.cc:3037] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:05.289711 23673 raft_consensus.cc:3037] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:05.292001 23673 raft_consensus.cc:2454] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f63946e9c17a4e2fa204f15c6e66aa75 in term 1.
I20260809 01:04:05.292004 23399 raft_consensus.cc:2454] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f63946e9c17a4e2fa204f15c6e66aa75 in term 1.
I20260809 01:04:05.292402 23469 leader_election.cc:302] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf3fb3a72e804928a2905e11e400bd77, f63946e9c17a4e2fa204f15c6e66aa75; no voters:
I20260809 01:04:05.292717 23747 raft_consensus.cc:2781] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:05.293160 23747 raft_consensus.cc:686] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [term 1 LEADER]: Becoming Leader. State: Replica: f63946e9c17a4e2fa204f15c6e66aa75, State: Running, Role: LEADER
I20260809 01:04:05.293649 23747 consensus_queue.cc:237] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef985d01c54d01865af07a542c065b" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45867 } } peers { permanent_uuid: "bf3fb3a72e804928a2905e11e400bd77" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 41207 } } peers { permanent_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 40335 } }
I20260809 01:04:05.298506 23252 catalog_manager.cc:5495] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f63946e9c17a4e2fa204f15c6e66aa75 (127.4.82.130). New cstate: current_term: 1 leader_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef985d01c54d01865af07a542c065b" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf3fb3a72e804928a2905e11e400bd77" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 41207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f63946e9c17a4e2fa204f15c6e66aa75" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 40335 } health_report { overall_health: HEALTHY } } }
I20260809 01:04:05.332521 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.327060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48182 (local address 127.4.82.130:40335)
0809 01:04:05.327279 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:05.327313 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:05.327417 (+ 104us) server_negotiation.cc:407] Connection header received
0809 01:04:05.327658 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:05.327662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:05.327755 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:05.327906 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:04:05.328267 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.328996 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.330392 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.330814 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.330949 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.331459 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:04:05.331943 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:04:05.332106 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:05.332166 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 01:04:05.332243 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 01:04:05.340225 23673 raft_consensus.cc:1261] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77 [term 1 FOLLOWER]: Refusing update from remote peer f63946e9c17a4e2fa204f15c6e66aa75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:04:05.340221 23399 raft_consensus.cc:1261] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b [term 1 FOLLOWER]: Refusing update from remote peer f63946e9c17a4e2fa204f15c6e66aa75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:04:05.341131 23751 consensus_queue.cc:1035] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf3fb3a72e804928a2905e11e400bd77" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 41207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:04:05.341487 23747 consensus_queue.cc:1035] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9ef985d01c54d01865af07a542c065b" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:04:05.351987 23755 mvcc.cc:205] Tried to move back new op lower bound from 7316428576103911424 to 7316428575927009280. Current Snapshot: MvccSnapshot[applied={T|T < 7316428576103911424}]
I20260809 01:04:05.353087 23757 mvcc.cc:205] Tried to move back new op lower bound from 7316428576103911424 to 7316428575927009280. Current Snapshot: MvccSnapshot[applied={T|T < 7316428576103911424}]
I20260809 01:04:05.366578 23251 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:04:05.366884 23251 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:04:05.370437 23251 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aeb0dadb653b4f9e8afd94319a125025: Sending DeleteTablet for 3 replicas of tablet bed00d37928d493f83899eb484ae1ae8
I20260809 01:04:05.371584 23517 tablet_service.cc:1514] Processing DeleteTablet for tablet bed00d37928d493f83899eb484ae1ae8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:05 UTC) from {username='slave'} at 127.0.0.1:48174
I20260809 01:04:05.371652 23651 tablet_service.cc:1514] Processing DeleteTablet for tablet bed00d37928d493f83899eb484ae1ae8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:05 UTC) from {username='slave'} at 127.0.0.1:52950
I20260809 01:04:05.371794 23378 tablet_service.cc:1514] Processing DeleteTablet for tablet bed00d37928d493f83899eb484ae1ae8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:05 UTC) from {username='slave'} at 127.0.0.1:54280
I20260809 01:04:05.373798 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 23314
I20260809 01:04:05.384258 23766 tablet_replica.cc:330] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75: stopping tablet replica
I20260809 01:04:05.384697 23766 raft_consensus.cc:2229] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:04:05.385321 23766 raft_consensus.cc:2258] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:05.386091 23767 tablet_replica.cc:330] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77: stopping tablet replica
I20260809 01:04:05.386464 23767 raft_consensus.cc:2229] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:04:05.386976 23767 raft_consensus.cc:2258] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:05.389174 23767 ts_tablet_manager.cc:1905] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:05.392597 23768 tablet_replica.cc:330] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b: stopping tablet replica
I20260809 01:04:05.393059 23768 raft_consensus.cc:2229] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:04:05.393652 23768 raft_consensus.cc:2258] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:05.395709 23766 ts_tablet_manager.cc:1905] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:05.395910 23768 ts_tablet_manager.cc:1905] T bed00d37928d493f83899eb484ae1ae8 P e9ef985d01c54d01865af07a542c065b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:05.400776 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.375541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54292 (local address 127.4.82.129:45867)
0809 01:04:05.375758 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:05.375767 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:05.391919 (+ 16152us) server_negotiation.cc:407] Connection header received
0809 01:04:05.392142 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:05.392150 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:05.392279 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:05.392399 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:05.393119 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.394302 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.396739 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.397430 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.397530 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.398984 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:05.398994 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:05.399009 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:05.399768 (+ 759us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:05.399829 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:05.399859 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:05.399952 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:05.400233 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 01:04:05.400346 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 01:04:05.401751 23443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:05.403008 23767 ts_tablet_manager.cc:1918] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:05.403096 23766 ts_tablet_manager.cc:1918] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:05.403301 23767 log.cc:1196] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/bed00d37928d493f83899eb484ae1ae8
I20260809 01:04:05.403302 23766 log.cc:1196] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/bed00d37928d493f83899eb484ae1ae8
I20260809 01:04:05.403882 23766 ts_tablet_manager.cc:1939] T bed00d37928d493f83899eb484ae1ae8 P f63946e9c17a4e2fa204f15c6e66aa75: Deleting consensus metadata
I20260809 01:04:05.403895 23767 ts_tablet_manager.cc:1939] T bed00d37928d493f83899eb484ae1ae8 P bf3fb3a72e804928a2905e11e400bd77: Deleting consensus metadata
I20260809 01:04:05.405628 23237 catalog_manager.cc:4841] TS f63946e9c17a4e2fa204f15c6e66aa75 (127.4.82.130:40335): tablet bed00d37928d493f83899eb484ae1ae8 (table test-table [id=341e4721acfd43718f5c544c03b454e9]) successfully deleted
I20260809 01:04:05.405786 23238 catalog_manager.cc:4841] TS bf3fb3a72e804928a2905e11e400bd77 (127.4.82.131:41207): tablet bed00d37928d493f83899eb484ae1ae8 (table test-table [id=341e4721acfd43718f5c544c03b454e9]) successfully deleted
I20260809 01:04:05.495142 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 23314
I20260809 01:04:05.516079 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 23452
W20260809 01:04:05.516041 23238 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.82.129:45867 (error 108)
I20260809 01:04:05.518493 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.517991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:45867 (local address 127.0.0.1:54298)
0809 01:04:05.518163 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.518318 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 01:04:05.519510 23238 catalog_manager.cc:4584] TS e9ef985d01c54d01865af07a542c065b (127.4.82.129:45867): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bed00d37928d493f83899eb484ae1ae8: Network error: Client connection negotiation failed: client connection to 127.4.82.129:45867: connect: Connection refused (error 111)
I20260809 01:04:05.524494 23736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.517251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.4.82.130:40335)
0809 01:04:05.517451 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:05.517456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:05.517472 (+ 16us) server_negotiation.cc:407] Connection header received
0809 01:04:05.517528 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:05.517531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:05.517621 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:05.517710 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:05.518275 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.519205 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.521203 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.521644 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.521736 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.522894 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:05.522902 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:05.522914 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:05.523592 (+ 678us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:05.523636 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:05.523662 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:05.523740 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:05.523926 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 01:04:05.524117 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 01:04:05.525285 23579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:05.573658 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.573031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:45867 (local address 127.0.0.1:54314)
0809 01:04:05.573318 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.573431 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 01:04:05.574951 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.574468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:45867 (local address 127.0.0.1:54328)
0809 01:04:05.574652 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.574759 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 01:04:05.635740 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.635143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:45867 (local address 127.0.0.1:54340)
0809 01:04:05.635446 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.635552 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 01:04:05.636781 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.636425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:45867 (local address 127.0.0.1:54356)
0809 01:04:05.636574 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.636652 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 01:04:05.640312 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 23452
I20260809 01:04:05.661252 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 23587
I20260809 01:04:05.668273 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.662047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52954 (local address 127.4.82.131:41207)
0809 01:04:05.662221 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:05.662226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:05.662244 (+ 18us) server_negotiation.cc:407] Connection header received
0809 01:04:05.662386 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:05.662390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:05.662467 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:05.662552 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:05.662919 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.663740 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.665049 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:05.665435 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:05.665522 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:05.666613 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:05.666622 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:05.666636 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:05.667201 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:05.667248 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:05.667267 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:05.667721 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:05.667859 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 01:04:05.667963 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 01:04:05.668920 23714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:05.717710 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.717091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:45867 (local address 127.0.0.1:54358)
0809 01:04:05.717408 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.717537 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 01:04:05.718813 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.718375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:45867 (local address 127.0.0.1:54366)
0809 01:04:05.718593 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.718680 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 01:04:05.769878 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 23587
I20260809 01:04:05.787840 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 23220
I20260809 01:04:05.788908 23284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:05.896123 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:05.895535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:45867 (local address 127.0.0.1:54374)
0809 01:04:05.895810 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:05.895922 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 01:04:05.896143 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 23220
2026-08-09T01:04:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2744 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:06 dist-test-slave-07c2 krb5kdc[23777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:06 dist-test-slave-07c2 krb5kdc[23777](info): set up 2 sockets
Aug 09 01:04:06 dist-test-slave-07c2 krb5kdc[23777](info): commencing operation
krb5kdc: starting...
W20260809 01:04:07.054745 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.001s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:07 dist-test-slave-07c2 krb5kdc[23777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:04:07.343101 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34633
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:34633
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:07.689905 23790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:07.690186 23790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:07.690276 23790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:07.697438 23790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:07.697548 23790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:07.697595 23790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:07.697660 23790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:07.697706 23790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:07.707438 23790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:34633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:07.708782 23790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:07.710021 23790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:07.722393 23797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:07.722438 23798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:07.722481 23800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:07.723248 23790 server_base.cc:1034] running on GCE node
I20260809 01:04:07.724084 23790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:07.725400 23790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:07.726624 23790 hybrid_clock.cc:648] HybridClock initialized: now 1786237447726585 us; error 34 us; skew 500 ppm
I20260809 01:04:07.726927 23790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:07 dist-test-slave-07c2 krb5kdc[23777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:07.730168 23790 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:04:07.731812 23790 webserver.cc:460] Webserver started at http://127.4.82.190:38475/ using document root <none> and password file <none>
I20260809 01:04:07.732504 23790 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:07.732643 23790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:07.732976 23790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:07.736038 23790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "0673efb9bea84a5484f70ae94e7951dd"
format_stamp: "Formatted at 2026-08-09 01:04:07 on dist-test-slave-07c2"
server_key: "0154c84a97bb66877ceb77ff62b10174"
server_key_iv: "b32ef01ac1fce57be71f708498bf2cfe"
server_key_version: "encryptionkey@0"
I20260809 01:04:07.736809 23790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "0673efb9bea84a5484f70ae94e7951dd"
format_stamp: "Formatted at 2026-08-09 01:04:07 on dist-test-slave-07c2"
server_key: "0154c84a97bb66877ceb77ff62b10174"
server_key_iv: "b32ef01ac1fce57be71f708498bf2cfe"
server_key_version: "encryptionkey@0"
I20260809 01:04:07.741933 23790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 01:04:07.745276 23807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:07.746505 23790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 01:04:07.746686 23790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "0673efb9bea84a5484f70ae94e7951dd"
format_stamp: "Formatted at 2026-08-09 01:04:07 on dist-test-slave-07c2"
server_key: "0154c84a97bb66877ceb77ff62b10174"
server_key_iv: "b32ef01ac1fce57be71f708498bf2cfe"
server_key_version: "encryptionkey@0"
I20260809 01:04:07.746860 23790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:07.763484 23790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:07.767776 23790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:07.768193 23790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:07.787348 23790 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:34633
I20260809 01:04:07.787420 23861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:34633 every 8 connection(s)
I20260809 01:04:07.788760 23790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:07.792335 23862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:07.795235 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 23790
I20260809 01:04:07.795470 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:07.796000 4426 external_mini_cluster.cc:1428] Setting key 2b7ee260bd914cad56c15dd5489b2b5e
I20260809 01:04:07.804669 23862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd: Bootstrap starting.
I20260809 01:04:07.808393 23862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:07.809733 23862 log.cc:824] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd: Log is configured to *not* fsync() on all Append() calls
Aug 09 01:04:07 dist-test-slave-07c2 krb5kdc[23777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:07.812970 23862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd: No bootstrap required, opened a new log
I20260809 01:04:07.820061 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:07.797934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35406 (local address 127.4.82.190:34633)
0809 01:04:07.798706 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:07.798717 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:07.798757 (+ 40us) server_negotiation.cc:407] Connection header received
0809 01:04:07.800502 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:07.800525 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:07.800915 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:07.801248 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:07.803147 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:07.804480 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:07.806356 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:07.807001 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:07.807134 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:07.812522 (+ 5388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:07.812530 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:07.812543 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:07.812584 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:07.815871 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:07.816695 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:07.816701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:07.816707 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:07.816886 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:07.817306 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:07.817309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:07.817311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:07.817822 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:07.818058 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:07.818559 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 01:04:07.818929 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":361,"threads_started":1}
I20260809 01:04:07.822084 23862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0673efb9bea84a5484f70ae94e7951dd" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34633 } }
I20260809 01:04:07.822438 23862 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:07.822574 23862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0673efb9bea84a5484f70ae94e7951dd, State: Initialized, Role: FOLLOWER
I20260809 01:04:07.823119 23862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0673efb9bea84a5484f70ae94e7951dd" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34633 } }
I20260809 01:04:07.823434 23862 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:07.823593 23862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:07.823768 23862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:07.825991 23862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0673efb9bea84a5484f70ae94e7951dd" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34633 } }
I20260809 01:04:07.826470 23862 leader_election.cc:302] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0673efb9bea84a5484f70ae94e7951dd; no voters:
I20260809 01:04:07.827044 23862 leader_election.cc:288] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:07.827193 23869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:07.827960 23869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [term 1 LEADER]: Becoming Leader. State: Replica: 0673efb9bea84a5484f70ae94e7951dd, State: Running, Role: LEADER
I20260809 01:04:07.828455 23869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0673efb9bea84a5484f70ae94e7951dd" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34633 } }
I20260809 01:04:07.829156 23862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:07.831329 23870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0673efb9bea84a5484f70ae94e7951dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0673efb9bea84a5484f70ae94e7951dd" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34633 } } }
I20260809 01:04:07.831707 23870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:07.832613 23871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0673efb9bea84a5484f70ae94e7951dd. Latest consensus state: current_term: 1 leader_uuid: "0673efb9bea84a5484f70ae94e7951dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0673efb9bea84a5484f70ae94e7951dd" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34633 } } }
I20260809 01:04:07.833058 23871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:07.837953 23874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:07.844131 23874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:07.852272 23874 catalog_manager.cc:1347] Generated new cluster ID: d77c1e86de654f2495f64cdeaee77fce
I20260809 01:04:07.852384 23874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:07.865347 23874 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:07.866526 23874 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:07.872687 23874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd: Generated new TSK 0
I20260809 01:04:07.873389 23874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 01:04:07.940807 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34633
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:08.315750 23890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:08.316021 23890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:08.316177 23890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:08.323437 23890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:08.323552 23890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:08.323750 23890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:04:08.333756 23890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:08.335016 23890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:08.336357 23890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:08.349555 23895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:08.349526 23896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:08.349583 23898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:08.350445 23890 server_base.cc:1034] running on GCE node
I20260809 01:04:08.351058 23890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:08.352137 23890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:08.353367 23890 hybrid_clock.cc:648] HybridClock initialized: now 1786237448353314 us; error 45 us; skew 500 ppm
I20260809 01:04:08.353672 23890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:08 dist-test-slave-07c2 krb5kdc[23777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:08.356863 23890 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:04:08.358232 23890 webserver.cc:460] Webserver started at http://127.4.82.129:44849/ using document root <none> and password file <none>
I20260809 01:04:08.358898 23890 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:08.359043 23890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:08.359421 23890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:08.362172 23890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "1c42fa610b264be39f286bea30bfe9fb"
format_stamp: "Formatted at 2026-08-09 01:04:08 on dist-test-slave-07c2"
server_key: "1b6bb7fafbb54ebfbff9dd6c4e5884fc"
server_key_iv: "5bc8cbcd14c199ef5ed986c6091e6c56"
server_key_version: "encryptionkey@0"
I20260809 01:04:08.362946 23890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "1c42fa610b264be39f286bea30bfe9fb"
format_stamp: "Formatted at 2026-08-09 01:04:08 on dist-test-slave-07c2"
server_key: "1b6bb7fafbb54ebfbff9dd6c4e5884fc"
server_key_iv: "5bc8cbcd14c199ef5ed986c6091e6c56"
server_key_version: "encryptionkey@0"
I20260809 01:04:08.368448 23890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 01:04:08.372433 23905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:08.373787 23890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 01:04:08.373952 23890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "1c42fa610b264be39f286bea30bfe9fb"
format_stamp: "Formatted at 2026-08-09 01:04:08 on dist-test-slave-07c2"
server_key: "1b6bb7fafbb54ebfbff9dd6c4e5884fc"
server_key_iv: "5bc8cbcd14c199ef5ed986c6091e6c56"
server_key_version: "encryptionkey@0"
I20260809 01:04:08.374154 23890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:08.407754 23890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:08.412005 23890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:08.412247 23890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:08.413147 23890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:08.414376 23890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:08.414449 23890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:08.414549 23890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:08.414594 23890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:08.448242 23890 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:37293
I20260809 01:04:08.448330 24024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:37293 every 8 connection(s)
I20260809 01:04:08.449566 23890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:04:08.454128 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 23890
I20260809 01:04:08.454324 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:04:08.454893 4426 external_mini_cluster.cc:1428] Setting key 31419dd0d19f649595d3f7466472aed6
Aug 09 01:04:08 dist-test-slave-07c2 krb5kdc[23777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:08.472779 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:08.452171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:52799 (local address 127.4.82.190:34633)
0809 01:04:08.452677 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:08.452683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:08.453606 (+ 923us) server_negotiation.cc:407] Connection header received
0809 01:04:08.456085 (+ 2479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:08.456091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:08.456183 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:08.456306 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:08.458179 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:08.458890 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:08.461361 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:08.461756 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:08.461852 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:08.465894 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:08.465903 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:08.465911 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:08.465958 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:08.469508 (+ 3550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:08.470442 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:08.470447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:08.470449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:08.470568 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:08.471062 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:08.471065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:08.471067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:08.471385 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:08.471496 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:08.472339 (+ 843us) server_negotiation.cc:299] Negotiation successful
0809 01:04:08.472500 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":225,"threads_started":1}
I20260809 01:04:08.474236 24028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:08.452742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34633 (local address 127.4.82.129:52799)
0809 01:04:08.453420 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:08.453457 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:08.455756 (+ 2299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:08.456498 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:08.456506 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:08.457203 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:08.457998 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:08.458026 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:08.459052 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:08.459060 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:08.461225 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:08.461242 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:08.461986 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:08.461996 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:08.462301 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:08.463013 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:04:08.463053 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:08.465663 (+ 2610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:08.469646 (+ 3983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:08.469652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:08.469663 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:08.470310 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:08.470678 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:08.470685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:08.470691 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:08.470946 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:08.471510 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:08.471520 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:08.471953 (+ 433us) client_negotiation.cc:769] Sending connection context
0809 01:04:08.472243 (+ 290us) client_negotiation.cc:240] Negotiation successful
0809 01:04:08.472801 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":273,"threads_started":1}
I20260809 01:04:08.475785 24025 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34633
I20260809 01:04:08.476083 24025 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:08.476770 24025 heartbeater.cc:511] Master 127.4.82.190:34633 requested a full tablet report, sending...
I20260809 01:04:08.478971 23825 ts_manager.cc:194] Registered new tserver with Master: 1c42fa610b264be39f286bea30bfe9fb (127.4.82.129:37293)
I20260809 01:04:08.480434 23825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:52799
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 01:04:08.520809 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34633
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:08.850994 24031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:08.851316 24031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:08.851490 24031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:08.858579 24031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:08.858718 24031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:08.858908 24031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:04:08.868358 24031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:08.869576 24031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:08.870848 24031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:08.883819 24037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:08.883946 24039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:08.884449 24036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:08.884960 24031 server_base.cc:1034] running on GCE node
I20260809 01:04:08.885506 24031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:08.886422 24031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:08.887650 24031 hybrid_clock.cc:648] HybridClock initialized: now 1786237448887606 us; error 50 us; skew 500 ppm
I20260809 01:04:08.887967 24031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:08 dist-test-slave-07c2 krb5kdc[23777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:08.894954 24031 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:04:08.896426 24031 webserver.cc:460] Webserver started at http://127.4.82.130:41481/ using document root <none> and password file <none>
I20260809 01:04:08.897073 24031 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:08.897224 24031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:08.897534 24031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:08.900360 24031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "1d337616185141259aa2065a903cd134"
format_stamp: "Formatted at 2026-08-09 01:04:08 on dist-test-slave-07c2"
server_key: "8e4d1c3051611a8f9938b54a3edf589c"
server_key_iv: "717c9d31e0667247d1a9df072b84bccf"
server_key_version: "encryptionkey@0"
I20260809 01:04:08.901144 24031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "1d337616185141259aa2065a903cd134"
format_stamp: "Formatted at 2026-08-09 01:04:08 on dist-test-slave-07c2"
server_key: "8e4d1c3051611a8f9938b54a3edf589c"
server_key_iv: "717c9d31e0667247d1a9df072b84bccf"
server_key_version: "encryptionkey@0"
I20260809 01:04:08.906097 24031 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 01:04:08.909426 24046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:08.910717 24031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 01:04:08.910892 24031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "1d337616185141259aa2065a903cd134"
format_stamp: "Formatted at 2026-08-09 01:04:08 on dist-test-slave-07c2"
server_key: "8e4d1c3051611a8f9938b54a3edf589c"
server_key_iv: "717c9d31e0667247d1a9df072b84bccf"
server_key_version: "encryptionkey@0"
I20260809 01:04:08.911093 24031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:08.928157 24031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:08.932179 24031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:08.932463 24031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:08.933399 24031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:08.934648 24031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:08.934763 24031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:08.934913 24031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:08.934994 24031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:08.968339 24031 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:45145
I20260809 01:04:08.968396 24164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:45145 every 8 connection(s)
I20260809 01:04:08.969679 24031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:04:08.979085 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 24031
I20260809 01:04:08.979279 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:04:08.979743 4426 external_mini_cluster.cc:1428] Setting key a467361a7b4b30a5b3129f6014f572b6
Aug 09 01:04:08 dist-test-slave-07c2 krb5kdc[23777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:08.989408 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:08.972201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:32943 (local address 127.4.82.190:34633)
0809 01:04:08.972382 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:08.972387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:08.973524 (+ 1137us) server_negotiation.cc:407] Connection header received
0809 01:04:08.975467 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:08.975473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:08.975570 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:08.975681 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:08.977145 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:08.977860 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:08.979572 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:08.980059 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:08.980196 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:08.983755 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:08.983760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:08.983763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:08.983795 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:08.986304 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:08.987023 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:08.987035 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:08.987038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:08.987177 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:08.987648 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:08.987651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:08.987653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:08.987922 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:08.988081 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:08.988825 (+ 744us) server_negotiation.cc:299] Negotiation successful
0809 01:04:08.989061 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 01:04:08.990376 24167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:08.972643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34633 (local address 127.4.82.130:32943)
0809 01:04:08.973363 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:08.973400 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:08.975099 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:08.975917 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:08.975926 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:08.976434 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:08.976971 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:08.976988 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:08.978024 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:08.978029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:08.979407 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:08.979423 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:08.980300 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:08.980308 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:08.980552 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:08.981230 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:04:08.981267 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:08.983533 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:08.986446 (+ 2913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:08.986454 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:08.986470 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:08.986891 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:08.987322 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:08.987326 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:08.987330 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:08.987517 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:08.988052 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:08.988060 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:08.988410 (+ 350us) client_negotiation.cc:769] Sending connection context
0809 01:04:08.988691 (+ 281us) client_negotiation.cc:240] Negotiation successful
0809 01:04:08.989237 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":282,"threads_started":1}
I20260809 01:04:08.992002 24165 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34633
I20260809 01:04:08.992311 24165 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:08.993013 24165 heartbeater.cc:511] Master 127.4.82.190:34633 requested a full tablet report, sending...
I20260809 01:04:08.994658 23825 ts_manager.cc:194] Registered new tserver with Master: 1d337616185141259aa2065a903cd134 (127.4.82.130:45145)
I20260809 01:04:08.995548 23825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:32943
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 01:04:09.043354 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34633
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:09.368286 24170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:09.368636 24170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:09.368821 24170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:09.377554 24170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:09.377719 24170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:09.377915 24170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:04:09.388386 24170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34633
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:09.389720 24170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:09.391018 24170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:09.403735 24175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:09.403782 24178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:09.404009 24176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:09.404631 24170 server_base.cc:1034] running on GCE node
I20260809 01:04:09.405190 24170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:09.406056 24170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:09.407315 24170 hybrid_clock.cc:648] HybridClock initialized: now 1786237449407233 us; error 60 us; skew 500 ppm
I20260809 01:04:09.407621 24170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:09 dist-test-slave-07c2 krb5kdc[23777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237449, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:09.410763 24170 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:04:09.412120 24170 webserver.cc:460] Webserver started at http://127.4.82.131:34937/ using document root <none> and password file <none>
I20260809 01:04:09.412730 24170 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:09.412830 24170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:09.413077 24170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:09.415946 24170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "37f1307a133146059f4b097a694227cc"
format_stamp: "Formatted at 2026-08-09 01:04:09 on dist-test-slave-07c2"
server_key: "46d518272a62e7edfdbab73af8643647"
server_key_iv: "fbf01efc06884af6d0524c0add196bcd"
server_key_version: "encryptionkey@0"
I20260809 01:04:09.416671 24170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "37f1307a133146059f4b097a694227cc"
format_stamp: "Formatted at 2026-08-09 01:04:09 on dist-test-slave-07c2"
server_key: "46d518272a62e7edfdbab73af8643647"
server_key_iv: "fbf01efc06884af6d0524c0add196bcd"
server_key_version: "encryptionkey@0"
I20260809 01:04:09.421558 24170 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 01:04:09.425153 24185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:09.426363 24170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 01:04:09.426487 24170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "37f1307a133146059f4b097a694227cc"
format_stamp: "Formatted at 2026-08-09 01:04:09 on dist-test-slave-07c2"
server_key: "46d518272a62e7edfdbab73af8643647"
server_key_iv: "fbf01efc06884af6d0524c0add196bcd"
server_key_version: "encryptionkey@0"
I20260809 01:04:09.426635 24170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:09.446100 24170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:09.450007 24170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:09.450306 24170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:09.451362 24170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:09.452684 24170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:09.452759 24170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:09.452895 24170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:09.452953 24170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:09.483246 24025 heartbeater.cc:503] Master 127.4.82.190:34633 was elected leader, sending a full tablet report...
I20260809 01:04:09.486697 24170 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:42295
I20260809 01:04:09.486781 24303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:42295 every 8 connection(s)
I20260809 01:04:09.487993 24170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:04:09.489194 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 24170
I20260809 01:04:09.489380 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:04:09.489943 4426 external_mini_cluster.cc:1428] Setting key 6cff320d0048cdc7d7909d10d24e1c6d
Aug 09 01:04:09 dist-test-slave-07c2 krb5kdc[23777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237449, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:09.508351 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.490731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:45267 (local address 127.4.82.190:34633)
0809 01:04:09.491249 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:09.491255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:09.492496 (+ 1241us) server_negotiation.cc:407] Connection header received
0809 01:04:09.494491 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:09.494496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:09.494595 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:09.494719 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:09.496219 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.497001 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.498541 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.498951 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.499087 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.502391 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:09.502404 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:09.502407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:09.502438 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:09.505773 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:09.506425 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:09.506428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:09.506431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:09.506551 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:09.506915 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:09.506918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:09.506920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:09.507156 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:09.507270 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:09.507935 (+ 665us) server_negotiation.cc:299] Negotiation successful
0809 01:04:09.508085 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":243,"threads_started":1}
I20260809 01:04:09.509238 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.491404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34633 (local address 127.4.82.131:45267)
0809 01:04:09.492348 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:09.492390 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:09.494242 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:09.494949 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:09.494956 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:09.495526 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:09.496046 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:09.496072 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.497177 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.497184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:09.498397 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:09.498409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.499181 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.499188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:09.499469 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.500058 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:04:09.500097 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:09.502185 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:09.505896 (+ 3711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:09.505903 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:09.505913 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:09.506318 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:09.506644 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:09.506647 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:09.506649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:09.506818 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:09.507260 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:09.507265 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:09.507597 (+ 332us) client_negotiation.cc:769] Sending connection context
0809 01:04:09.507828 (+ 231us) client_negotiation.cc:240] Negotiation successful
0809 01:04:09.508168 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":476,"threads_started":1}
I20260809 01:04:09.510509 24304 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34633
I20260809 01:04:09.510803 24304 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:09.511535 24304 heartbeater.cc:511] Master 127.4.82.190:34633 requested a full tablet report, sending...
I20260809 01:04:09.512814 23825 ts_manager.cc:194] Registered new tserver with Master: 37f1307a133146059f4b097a694227cc (127.4.82.131:42295)
I20260809 01:04:09.513595 23825 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:45267
I20260809 01:04:09.517463 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:04:09.534516 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.521773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56024 (local address 127.4.82.190:34633)
0809 01:04:09.521924 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:09.521929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:09.522455 (+ 526us) server_negotiation.cc:407] Connection header received
0809 01:04:09.522583 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:09.522586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:09.522644 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:09.522722 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:09.524092 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.524810 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.526144 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.526518 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.526608 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.529172 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:09.529177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:09.529180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:09.529227 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:09.532023 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:09.532701 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:09.532704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:09.532706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:09.532820 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:09.533188 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:09.533191 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:09.533193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:09.533481 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:09.533680 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:09.534139 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 01:04:09.534284 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 01:04:09.538414 23825 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:04:09.545897 23825 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:04:09.568535 24319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.557628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43454 (local address 127.4.82.130:45145)
0809 01:04:09.559671 (+ 2043us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:09.559679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:09.559705 (+ 26us) server_negotiation.cc:407] Connection header received
0809 01:04:09.559816 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:09.559823 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:09.560023 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:09.561202 (+ 1179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:09.561443 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.563098 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.565617 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.567096 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.567248 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.567493 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:09.568018 (+ 525us) server_negotiation.cc:299] Negotiation successful
0809 01:04:09.568195 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1905,"thread_start_us":1774,"threads_started":1}
I20260809 01:04:09.568563 24320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.557768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:37293 (local address 127.0.0.1:39004)
0809 01:04:09.559051 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:09.559068 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:09.559172 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:09.559996 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:09.560003 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:09.560032 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:09.560598 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:09.560616 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.563230 (+ 2614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.563236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:09.565678 (+ 2442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:09.565692 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.567518 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.567526 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:09.567733 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.567765 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 01:04:09.567842 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 01:04:09.568065 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1154,"mutex_wait_us":95,"thread_start_us":1442,"threads_started":1}
I20260809 01:04:09.569253 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.557436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:45145 (local address 127.0.0.1:43454)
0809 01:04:09.558478 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:09.558512 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:09.558670 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:09.560246 (+ 1576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:09.560251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:09.560272 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:09.561307 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:09.561324 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.563322 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.563329 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:09.565466 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:09.565478 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.567326 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.567334 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:09.567627 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.567695 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 01:04:09.567844 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 01:04:09.568837 (+ 993us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":933,"thread_start_us":673,"threads_started":1}
I20260809 01:04:09.569478 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.560100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.4.82.131:42295)
0809 01:04:09.561093 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:09.561102 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:09.561701 (+ 599us) server_negotiation.cc:407] Connection header received
0809 01:04:09.561801 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:09.561809 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:09.562015 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:09.562205 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:09.562893 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.564517 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.566642 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.567802 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.567917 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.568132 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:09.568985 (+ 853us) server_negotiation.cc:299] Negotiation successful
0809 01:04:09.569143 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":626,"threads_started":1}
I20260809 01:04:09.569964 24321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.558575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39004 (local address 127.4.82.129:37293)
0809 01:04:09.559348 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:09.559360 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:09.559392 (+ 32us) server_negotiation.cc:407] Connection header received
0809 01:04:09.559496 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:09.559504 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:09.559762 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:09.560384 (+ 622us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:09.561520 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.563076 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.565853 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.567265 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.569162 (+ 1897us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.569424 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:09.569506 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 01:04:09.569670 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":1123,"threads_started":1}
I20260809 01:04:09.570434 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.559579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:42295 (local address 127.0.0.1:54788)
0809 01:04:09.561443 (+ 1864us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:09.561460 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:09.561577 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:09.562424 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:09.562427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:09.562446 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:09.562708 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:09.562715 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.564723 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.564729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:09.566487 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:09.566499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.568305 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.568313 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:09.568498 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.568559 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 01:04:09.568659 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 01:04:09.569068 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1743,"mutex_wait_us":2,"thread_start_us":1566,"threads_started":1}
I20260809 01:04:09.573300 24096 tablet_service.cc:1467] Processing CreateTablet for tablet cf129ae95cdc42dfb76f5c565380e7f0 (DEFAULT_TABLE table=test-table [id=975b55e398cf4628a18ef57e2ddad74e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:09.573300 23955 tablet_service.cc:1467] Processing CreateTablet for tablet cf129ae95cdc42dfb76f5c565380e7f0 (DEFAULT_TABLE table=test-table [id=975b55e398cf4628a18ef57e2ddad74e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:09.573786 24235 tablet_service.cc:1467] Processing CreateTablet for tablet cf129ae95cdc42dfb76f5c565380e7f0 (DEFAULT_TABLE table=test-table [id=975b55e398cf4628a18ef57e2ddad74e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:09.574836 23955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf129ae95cdc42dfb76f5c565380e7f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:09.574836 24096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf129ae95cdc42dfb76f5c565380e7f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:09.575218 24235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf129ae95cdc42dfb76f5c565380e7f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:09.588320 24324 tablet_bootstrap.cc:492] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb: Bootstrap starting.
I20260809 01:04:09.592080 24324 tablet_bootstrap.cc:654] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:09.593096 24324 log.cc:824] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:09.594095 24326 tablet_bootstrap.cc:492] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134: Bootstrap starting.
I20260809 01:04:09.595988 24325 tablet_bootstrap.cc:492] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc: Bootstrap starting.
I20260809 01:04:09.597767 24324 tablet_bootstrap.cc:492] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb: No bootstrap required, opened a new log
I20260809 01:04:09.598011 24324 ts_tablet_manager.cc:1397] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260809 01:04:09.598218 24326 tablet_bootstrap.cc:654] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:09.599472 24326 log.cc:824] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:09.600431 24325 tablet_bootstrap.cc:654] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:09.601647 24325 log.cc:824] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:09.602536 24326 tablet_bootstrap.cc:492] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134: No bootstrap required, opened a new log
I20260809 01:04:09.602804 24326 ts_tablet_manager.cc:1397] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 01:04:09.604640 24325 tablet_bootstrap.cc:492] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc: No bootstrap required, opened a new log
I20260809 01:04:09.604893 24325 ts_tablet_manager.cc:1397] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 01:04:09.605934 24324 raft_consensus.cc:348] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d337616185141259aa2065a903cd134" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45145 } } peers { permanent_uuid: "1c42fa610b264be39f286bea30bfe9fb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37293 } } peers { permanent_uuid: "37f1307a133146059f4b097a694227cc" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42295 } }
I20260809 01:04:09.606480 24324 raft_consensus.cc:374] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:09.606653 24324 raft_consensus.cc:729] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c42fa610b264be39f286bea30bfe9fb, State: Initialized, Role: FOLLOWER
I20260809 01:04:09.607396 24324 consensus_queue.cc:260] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d337616185141259aa2065a903cd134" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45145 } } peers { permanent_uuid: "1c42fa610b264be39f286bea30bfe9fb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37293 } } peers { permanent_uuid: "37f1307a133146059f4b097a694227cc" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42295 } }
I20260809 01:04:09.608968 24324 ts_tablet_manager.cc:1428] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 01:04:09.613745 24326 raft_consensus.cc:348] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d337616185141259aa2065a903cd134" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45145 } } peers { permanent_uuid: "1c42fa610b264be39f286bea30bfe9fb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37293 } } peers { permanent_uuid: "37f1307a133146059f4b097a694227cc" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42295 } }
I20260809 01:04:09.613729 24325 raft_consensus.cc:348] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d337616185141259aa2065a903cd134" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45145 } } peers { permanent_uuid: "1c42fa610b264be39f286bea30bfe9fb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37293 } } peers { permanent_uuid: "37f1307a133146059f4b097a694227cc" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42295 } }
I20260809 01:04:09.614099 24326 raft_consensus.cc:374] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:09.614199 24325 raft_consensus.cc:374] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:09.614223 24326 raft_consensus.cc:729] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d337616185141259aa2065a903cd134, State: Initialized, Role: FOLLOWER
I20260809 01:04:09.614308 24325 raft_consensus.cc:729] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37f1307a133146059f4b097a694227cc, State: Initialized, Role: FOLLOWER
I20260809 01:04:09.614965 24325 consensus_queue.cc:260] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d337616185141259aa2065a903cd134" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45145 } } peers { permanent_uuid: "1c42fa610b264be39f286bea30bfe9fb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37293 } } peers { permanent_uuid: "37f1307a133146059f4b097a694227cc" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42295 } }
I20260809 01:04:09.614981 24326 consensus_queue.cc:260] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d337616185141259aa2065a903cd134" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45145 } } peers { permanent_uuid: "1c42fa610b264be39f286bea30bfe9fb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37293 } } peers { permanent_uuid: "37f1307a133146059f4b097a694227cc" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42295 } }
I20260809 01:04:09.616117 24304 heartbeater.cc:503] Master 127.4.82.190:34633 was elected leader, sending a full tablet report...
I20260809 01:04:09.616560 24165 heartbeater.cc:503] Master 127.4.82.190:34633 was elected leader, sending a full tablet report...
I20260809 01:04:09.616758 24325 ts_tablet_manager.cc:1428] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc: Time spent starting tablet: real 0.012s user 0.011s sys 0.002s
I20260809 01:04:09.617127 24326 ts_tablet_manager.cc:1428] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134: Time spent starting tablet: real 0.014s user 0.011s sys 0.003s
I20260809 01:04:09.679648 24332 raft_consensus.cc:482] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:04:09.679979 24332 raft_consensus.cc:504] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d337616185141259aa2065a903cd134" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45145 } } peers { permanent_uuid: "1c42fa610b264be39f286bea30bfe9fb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37293 } } peers { permanent_uuid: "37f1307a133146059f4b097a694227cc" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42295 } }
I20260809 01:04:09.681648 24332 leader_election.cc:288] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c42fa610b264be39f286bea30bfe9fb (127.4.82.129:37293), 37f1307a133146059f4b097a694227cc (127.4.82.131:42295)
I20260809 01:04:09.689577 24333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.682033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:37293 (local address 127.4.82.130:47289)
0809 01:04:09.682684 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:09.682705 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:09.682905 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:09.683560 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:09.683567 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:09.683603 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:09.684117 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:09.684130 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.685479 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.685482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:09.687534 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:09.687549 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.688922 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.688927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:09.689041 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.689065 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 01:04:09.689141 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 01:04:09.689237 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":376,"threads_started":1}
I20260809 01:04:09.689600 24321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.682159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:47289 (local address 127.4.82.129:37293)
0809 01:04:09.682445 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:09.682453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:09.682849 (+ 396us) server_negotiation.cc:407] Connection header received
0809 01:04:09.683104 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:09.683110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:09.683215 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:09.683315 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:09.684305 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.685346 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.687684 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.688700 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.688886 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.688962 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:09.689195 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 01:04:09.689335 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 01:04:09.690353 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.682834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:42295 (local address 127.4.82.130:56647)
0809 01:04:09.684191 (+ 1357us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:09.684210 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:09.684370 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:09.684740 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:09.684742 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:09.684761 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:09.685039 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:09.685046 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.686129 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.686134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:09.688007 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:09.688017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.689561 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.689573 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:09.689783 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.689814 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 01:04:09.689903 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 01:04:09.690026 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1170,"thread_start_us":871,"threads_started":1}
I20260809 01:04:09.690361 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.683096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:56647 (local address 127.4.82.131:42295)
0809 01:04:09.683367 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:09.683373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:09.684280 (+ 907us) server_negotiation.cc:407] Connection header received
0809 01:04:09.684517 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:09.684523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:09.684601 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:09.684712 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:09.685182 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.685994 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.688149 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.689335 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.689493 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.689555 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:09.689952 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 01:04:09.690097 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 01:04:09.690888 23976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf129ae95cdc42dfb76f5c565380e7f0" candidate_uuid: "1d337616185141259aa2065a903cd134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c42fa610b264be39f286bea30bfe9fb" is_pre_election: true
I20260809 01:04:09.691524 23976 raft_consensus.cc:2454] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d337616185141259aa2065a903cd134 in term 0.
I20260809 01:04:09.691721 24256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf129ae95cdc42dfb76f5c565380e7f0" candidate_uuid: "1d337616185141259aa2065a903cd134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37f1307a133146059f4b097a694227cc" is_pre_election: true
I20260809 01:04:09.692238 24256 raft_consensus.cc:2454] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d337616185141259aa2065a903cd134 in term 0.
I20260809 01:04:09.692242 24049 leader_election.cc:302] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c42fa610b264be39f286bea30bfe9fb, 1d337616185141259aa2065a903cd134; no voters:
I20260809 01:04:09.692667 24332 raft_consensus.cc:2781] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:04:09.692823 24332 raft_consensus.cc:482] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:04:09.692955 24332 raft_consensus.cc:3037] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:09.695264 24332 raft_consensus.cc:504] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d337616185141259aa2065a903cd134" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45145 } } peers { permanent_uuid: "1c42fa610b264be39f286bea30bfe9fb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37293 } } peers { permanent_uuid: "37f1307a133146059f4b097a694227cc" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42295 } }
I20260809 01:04:09.695931 24332 leader_election.cc:288] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [CANDIDATE]: Term 1 election: Requested vote from peers 1c42fa610b264be39f286bea30bfe9fb (127.4.82.129:37293), 37f1307a133146059f4b097a694227cc (127.4.82.131:42295)
I20260809 01:04:09.696358 23976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf129ae95cdc42dfb76f5c565380e7f0" candidate_uuid: "1d337616185141259aa2065a903cd134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c42fa610b264be39f286bea30bfe9fb"
I20260809 01:04:09.696444 24256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf129ae95cdc42dfb76f5c565380e7f0" candidate_uuid: "1d337616185141259aa2065a903cd134" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37f1307a133146059f4b097a694227cc"
I20260809 01:04:09.696580 23976 raft_consensus.cc:3037] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:09.696628 24256 raft_consensus.cc:3037] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:09.699889 23976 raft_consensus.cc:2454] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d337616185141259aa2065a903cd134 in term 1.
I20260809 01:04:09.699889 24256 raft_consensus.cc:2454] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d337616185141259aa2065a903cd134 in term 1.
I20260809 01:04:09.700316 24049 leader_election.cc:302] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c42fa610b264be39f286bea30bfe9fb, 1d337616185141259aa2065a903cd134; no voters:
I20260809 01:04:09.700623 24332 raft_consensus.cc:2781] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:09.701092 24332 raft_consensus.cc:686] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [term 1 LEADER]: Becoming Leader. State: Replica: 1d337616185141259aa2065a903cd134, State: Running, Role: LEADER
I20260809 01:04:09.701586 24332 consensus_queue.cc:237] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d337616185141259aa2065a903cd134" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45145 } } peers { permanent_uuid: "1c42fa610b264be39f286bea30bfe9fb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37293 } } peers { permanent_uuid: "37f1307a133146059f4b097a694227cc" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42295 } }
W20260809 01:04:09.702005 24026 tablet.cc:2369] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:04:09.706660 23825 catalog_manager.cc:5495] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d337616185141259aa2065a903cd134 (127.4.82.130). New cstate: current_term: 1 leader_uuid: "1d337616185141259aa2065a903cd134" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d337616185141259aa2065a903cd134" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 45145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c42fa610b264be39f286bea30bfe9fb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37f1307a133146059f4b097a694227cc" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42295 } health_report { overall_health: UNKNOWN } } }
W20260809 01:04:09.721786 24166 tablet.cc:2369] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:04:09.733596 24319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.728029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43462 (local address 127.4.82.130:45145)
0809 01:04:09.728211 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:09.728215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:09.728233 (+ 18us) server_negotiation.cc:407] Connection header received
0809 01:04:09.728361 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:09.728365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:09.728437 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:09.728573 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:04:09.728941 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.729758 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.731579 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.731974 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.732068 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.732532 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:04:09.733021 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:04:09.733149 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:09.733253 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 01:04:09.733314 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 01:04:09.739543 24305 tablet.cc:2369] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:04:09.741654 24256 raft_consensus.cc:1261] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc [term 1 FOLLOWER]: Refusing update from remote peer 1d337616185141259aa2065a903cd134: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:04:09.741824 23976 raft_consensus.cc:1261] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb [term 1 FOLLOWER]: Refusing update from remote peer 1d337616185141259aa2065a903cd134: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:04:09.742705 24332 consensus_queue.cc:1035] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37f1307a133146059f4b097a694227cc" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 42295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:04:09.743040 24335 consensus_queue.cc:1035] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c42fa610b264be39f286bea30bfe9fb" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 37293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:04:09.770864 23823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:04:09.771217 23823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:04:09.775462 23823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0673efb9bea84a5484f70ae94e7951dd: Sending DeleteTablet for 3 replicas of tablet cf129ae95cdc42dfb76f5c565380e7f0
I20260809 01:04:09.776746 24096 tablet_service.cc:1514] Processing DeleteTablet for tablet cf129ae95cdc42dfb76f5c565380e7f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:09 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:43454
I20260809 01:04:09.776875 24235 tablet_service.cc:1514] Processing DeleteTablet for tablet cf129ae95cdc42dfb76f5c565380e7f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:09 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:54788
I20260809 01:04:09.777093 23955 tablet_service.cc:1514] Processing DeleteTablet for tablet cf129ae95cdc42dfb76f5c565380e7f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:09 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:39004
I20260809 01:04:09.782506 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 23890
I20260809 01:04:09.804929 24351 tablet_replica.cc:330] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb: stopping tablet replica
I20260809 01:04:09.805460 24351 raft_consensus.cc:2229] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:04:09.806028 24351 raft_consensus.cc:2258] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:09.806967 24350 tablet_replica.cc:330] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc: stopping tablet replica
I20260809 01:04:09.807245 24349 tablet_replica.cc:330] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134: stopping tablet replica
I20260809 01:04:09.807400 24350 raft_consensus.cc:2229] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:04:09.807713 24349 raft_consensus.cc:2229] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:04:09.808002 24350 raft_consensus.cc:2258] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:09.808513 24349 raft_consensus.cc:2258] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:09.809413 24351 ts_tablet_manager.cc:1905] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:09.810554 24349 ts_tablet_manager.cc:1905] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:09.815402 24350 ts_tablet_manager.cc:1905] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:09.818202 24349 ts_tablet_manager.cc:1918] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:09.818416 24349 log.cc:1196] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/cf129ae95cdc42dfb76f5c565380e7f0
Aug 09 01:04:09 dist-test-slave-07c2 krb5kdc[23777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 01:04:09.819064 24349 ts_tablet_manager.cc:1939] T cf129ae95cdc42dfb76f5c565380e7f0 P 1d337616185141259aa2065a903cd134: Deleting consensus metadata
I20260809 01:04:09.820859 23812 catalog_manager.cc:4841] TS 1d337616185141259aa2065a903cd134 (127.4.82.130:45145): tablet cf129ae95cdc42dfb76f5c565380e7f0 (table test-table [id=975b55e398cf4628a18ef57e2ddad74e]) successfully deleted
I20260809 01:04:09.822773 24350 ts_tablet_manager.cc:1918] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:09.826187 24351 ts_tablet_manager.cc:1918] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:09.826427 24351 log.cc:1196] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/cf129ae95cdc42dfb76f5c565380e7f0
I20260809 01:04:09.827059 24351 ts_tablet_manager.cc:1939] T cf129ae95cdc42dfb76f5c565380e7f0 P 1c42fa610b264be39f286bea30bfe9fb: Deleting consensus metadata
I20260809 01:04:09.828016 24350 log.cc:1196] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/cf129ae95cdc42dfb76f5c565380e7f0
I20260809 01:04:09.828639 24350 ts_tablet_manager.cc:1939] T cf129ae95cdc42dfb76f5c565380e7f0 P 37f1307a133146059f4b097a694227cc: Deleting consensus metadata
I20260809 01:04:09.829214 23810 catalog_manager.cc:4841] TS 1c42fa610b264be39f286bea30bfe9fb (127.4.82.129:37293): tablet cf129ae95cdc42dfb76f5c565380e7f0 (table test-table [id=975b55e398cf4628a18ef57e2ddad74e]) successfully deleted
I20260809 01:04:09.830381 24321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.784286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39016 (local address 127.4.82.129:37293)
0809 01:04:09.784512 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:09.784522 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:09.806778 (+ 22256us) server_negotiation.cc:407] Connection header received
0809 01:04:09.807471 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:09.807477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:09.807561 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:09.807642 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:09.810273 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.811352 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.813346 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.814002 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.814092 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.822384 (+ 8292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:09.822398 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:09.822421 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:09.822470 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:09.826630 (+ 4160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:09.827624 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:09.827633 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:09.827636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:09.827813 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:09.828308 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:09.828312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:09.828315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:09.828780 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:09.829068 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:09.829742 (+ 674us) server_negotiation.cc:299] Negotiation successful
0809 01:04:09.829975 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 01:04:09.830394 23810 catalog_manager.cc:4841] TS 37f1307a133146059f4b097a694227cc (127.4.82.131:42295): tablet cf129ae95cdc42dfb76f5c565380e7f0 (table test-table [id=975b55e398cf4628a18ef57e2ddad74e]) successfully deleted
I20260809 01:04:09.831532 24020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:09.920241 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 23890
I20260809 01:04:09.940728 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 24031
Aug 09 01:04:09 dist-test-slave-07c2 krb5kdc[23777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 01:04:09.958339 24319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:09.941670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43464 (local address 127.4.82.130:45145)
0809 01:04:09.941879 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:09.941886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:09.941906 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:04:09.941984 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:09.941995 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:09.942072 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:09.942192 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:09.943367 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.944631 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.945895 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:09.946605 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:09.946733 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:09.951361 (+ 4628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:09.951383 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:09.951403 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:09.951455 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:09.955451 (+ 3996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:09.956110 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:09.956119 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:09.956123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:09.956326 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:09.956694 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:09.956700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:09.956703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:09.957145 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:09.957505 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:09.957673 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 01:04:09.957895 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 01:04:09.959190 24160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:10.071446 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 24031
I20260809 01:04:10.092871 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 24170
Aug 09 01:04:10 dist-test-slave-07c2 krb5kdc[23777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 01:04:10.108611 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:10.093626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54794 (local address 127.4.82.131:42295)
0809 01:04:10.093831 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:10.093837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:10.093907 (+ 70us) server_negotiation.cc:407] Connection header received
0809 01:04:10.094132 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:10.094136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:10.094200 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:10.094315 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:10.095720 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:10.096448 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:10.097715 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:10.098121 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:10.098255 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:10.102887 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:10.102901 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:10.102917 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:10.102950 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:10.106149 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:10.106722 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:10.106729 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:10.106731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:10.106866 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:10.107192 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:10.107195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:10.107197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:10.107597 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:10.107824 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:10.108071 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 01:04:10.108243 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 01:04:10.109266 24299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:10.197773 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 24170
I20260809 01:04:10.216714 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 23790
I20260809 01:04:10.217974 23857 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:10.329460 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 23790
2026-08-09T01:04:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4434 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T01:04:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:10Z Disabled control of system clock
I20260809 01:04:10.416800 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:46225
--webserver_interface=127.4.82.190
--webserver_port=0
--builtin_ntp_servers=127.4.82.148:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:46225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 01:04:10.764338 24360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:10.764603 24360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:10.764693 24360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:10.771685 24360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:10.771783 24360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:10.771835 24360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:10.771891 24360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:10.771929 24360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:10.780742 24360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:46225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:46225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:10.781883 24360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:10.783594 24360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:10.795127 24366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:10.795214 24365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:10.795228 24368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:10.800738 24360 server_base.cc:1034] running on GCE node
I20260809 01:04:10.801540 24360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:10.802937 24360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:10.804177 24360 hybrid_clock.cc:648] HybridClock initialized: now 1786237450804135 us; error 53 us; skew 500 ppm
I20260809 01:04:10.804564 24360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:04:10.806802 24360 webserver.cc:460] Webserver started at http://127.4.82.190:36613/ using document root <none> and password file <none>
I20260809 01:04:10.807551 24360 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:10.807682 24360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:10.808003 24360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:10.810870 24360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "fc4d59a3a0a74c79a55b474f45180cb1"
format_stamp: "Formatted at 2026-08-09 01:04:10 on dist-test-slave-07c2"
server_key: "b7920532aea0db4c3e9a6a919f68f4b1"
server_key_iv: "fef19900ae8226d4876767b1e05b3fe4"
server_key_version: "encryptionkey@0"
I20260809 01:04:10.811750 24360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "fc4d59a3a0a74c79a55b474f45180cb1"
format_stamp: "Formatted at 2026-08-09 01:04:10 on dist-test-slave-07c2"
server_key: "b7920532aea0db4c3e9a6a919f68f4b1"
server_key_iv: "fef19900ae8226d4876767b1e05b3fe4"
server_key_version: "encryptionkey@0"
I20260809 01:04:10.816582 24360 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 01:04:10.819864 24374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:10.821245 24360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 01:04:10.821439 24360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "fc4d59a3a0a74c79a55b474f45180cb1"
format_stamp: "Formatted at 2026-08-09 01:04:10 on dist-test-slave-07c2"
server_key: "b7920532aea0db4c3e9a6a919f68f4b1"
server_key_iv: "fef19900ae8226d4876767b1e05b3fe4"
server_key_version: "encryptionkey@0"
I20260809 01:04:10.821650 24360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:10.834196 24360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:10.834939 24360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:10.835215 24360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:10.853559 24360 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:46225
I20260809 01:04:10.853588 24427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:46225 every 8 connection(s)
I20260809 01:04:10.854857 24360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:10.858376 24428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:10.860863 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 24360
I20260809 01:04:10.861040 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:10.861598 4426 external_mini_cluster.cc:1428] Setting key 9db82f18848af16614b040bbb542de9b
I20260809 01:04:10.869810 24428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1: Bootstrap starting.
I20260809 01:04:10.872967 24428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:10.874186 24428 log.cc:824] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:10.874902 24431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:10.863604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.4.82.190:46225)
0809 01:04:10.864597 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:10.864607 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:10.864644 (+ 37us) server_negotiation.cc:407] Connection header received
0809 01:04:10.866279 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:10.866299 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:10.866710 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:10.867099 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:10.867693 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:10.868967 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:10.870350 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:10.871002 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:10.871207 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:10.872442 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:10.872449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:10.872462 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:10.872961 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:10.873053 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:10.873094 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:10.873209 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:10.873497 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 01:04:10.873803 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":375,"threads_started":1}
I20260809 01:04:10.877106 24428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1: No bootstrap required, opened a new log
I20260809 01:04:10.884363 24428 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4d59a3a0a74c79a55b474f45180cb1" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46225 } }
I20260809 01:04:10.884716 24428 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:10.884833 24428 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc4d59a3a0a74c79a55b474f45180cb1, State: Initialized, Role: FOLLOWER
I20260809 01:04:10.885354 24428 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4d59a3a0a74c79a55b474f45180cb1" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46225 } }
I20260809 01:04:10.885593 24428 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:10.885732 24428 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:10.885924 24428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:10.888042 24428 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4d59a3a0a74c79a55b474f45180cb1" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46225 } }
I20260809 01:04:10.888521 24428 leader_election.cc:302] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc4d59a3a0a74c79a55b474f45180cb1; no voters:
I20260809 01:04:10.889123 24428 leader_election.cc:288] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:10.889272 24433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:10.889982 24433 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [term 1 LEADER]: Becoming Leader. State: Replica: fc4d59a3a0a74c79a55b474f45180cb1, State: Running, Role: LEADER
I20260809 01:04:10.890579 24433 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4d59a3a0a74c79a55b474f45180cb1" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46225 } }
I20260809 01:04:10.891108 24428 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:10.893400 24434 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc4d59a3a0a74c79a55b474f45180cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4d59a3a0a74c79a55b474f45180cb1" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46225 } } }
I20260809 01:04:10.893339 24435 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc4d59a3a0a74c79a55b474f45180cb1. Latest consensus state: current_term: 1 leader_uuid: "fc4d59a3a0a74c79a55b474f45180cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc4d59a3a0a74c79a55b474f45180cb1" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46225 } } }
I20260809 01:04:10.893867 24434 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:10.893896 24435 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:10.897924 24441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:10.901995 24441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:10.909096 24441 catalog_manager.cc:1347] Generated new cluster ID: 6c4799cf7bf14d2298b23b47fa3c860f
I20260809 01:04:10.909199 24441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:10.938956 24441 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:10.940763 24441 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:10.956621 24441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1: Generated new TSK 0
I20260809 01:04:10.957435 24441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:10.973071 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46225
--builtin_ntp_servers=127.4.82.148:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 01:04:11.307354 24454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:11.307616 24454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:11.307711 24454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:11.314666 24454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:11.314751 24454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:11.314857 24454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:04:11.323791 24454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46225
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:11.325016 24454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:11.326815 24454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:11.340188 24459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:11.340342 24454 server_base.cc:1034] running on GCE node
W20260809 01:04:11.340197 24460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:11.340164 24462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:11.341286 24454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:11.342088 24454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:11.343327 24454 hybrid_clock.cc:648] HybridClock initialized: now 1786237451343234 us; error 53 us; skew 500 ppm
I20260809 01:04:11.343595 24454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:04:11.345638 24454 webserver.cc:460] Webserver started at http://127.4.82.129:34397/ using document root <none> and password file <none>
I20260809 01:04:11.346246 24454 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:11.346331 24454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:11.346580 24454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:11.349251 24454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "625fd4313c28452fb01b54d5915bf09a"
format_stamp: "Formatted at 2026-08-09 01:04:11 on dist-test-slave-07c2"
server_key: "61bbddd929db3702110998626845dae0"
server_key_iv: "e10698fed51e3ec5e498f6366643dbeb"
server_key_version: "encryptionkey@0"
I20260809 01:04:11.349943 24454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "625fd4313c28452fb01b54d5915bf09a"
format_stamp: "Formatted at 2026-08-09 01:04:11 on dist-test-slave-07c2"
server_key: "61bbddd929db3702110998626845dae0"
server_key_iv: "e10698fed51e3ec5e498f6366643dbeb"
server_key_version: "encryptionkey@0"
I20260809 01:04:11.354621 24454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260809 01:04:11.357795 24468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:11.358963 24454 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 01:04:11.359088 24454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "625fd4313c28452fb01b54d5915bf09a"
format_stamp: "Formatted at 2026-08-09 01:04:11 on dist-test-slave-07c2"
server_key: "61bbddd929db3702110998626845dae0"
server_key_iv: "e10698fed51e3ec5e498f6366643dbeb"
server_key_version: "encryptionkey@0"
I20260809 01:04:11.359259 24454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:11.371112 24454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:11.371870 24454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:11.372107 24454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:11.372931 24454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:11.374269 24454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:11.374354 24454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:11.374431 24454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:11.374492 24454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:11.408913 24454 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:46849
I20260809 01:04:11.408986 24585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:46849 every 8 connection(s)
I20260809 01:04:11.410169 24454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:04:11.411893 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 24454
I20260809 01:04:11.412072 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:04:11.412569 4426 external_mini_cluster.cc:1428] Setting key 4b91f7f303f11d283b23b248426ff0ca
I20260809 01:04:11.415578 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46225
--builtin_ntp_servers=127.4.82.148:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 01:04:11.422715 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:11.413162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:56929 (local address 127.4.82.190:46225)
0809 01:04:11.413628 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:11.413634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:11.415378 (+ 1744us) server_negotiation.cc:407] Connection header received
0809 01:04:11.416275 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:11.416280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:11.416371 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:11.416486 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:11.417517 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:11.418264 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:11.419850 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:11.420259 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:11.420394 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:11.421734 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:11.421738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:11.421742 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:11.421759 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:11.421806 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:11.421818 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:11.421950 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:11.422267 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 01:04:11.422367 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":246,"threads_started":1}
I20260809 01:04:11.423717 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:11.413170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:46225 (local address 127.4.82.129:56929)
0809 01:04:11.414270 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:11.414315 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:11.416016 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:11.416724 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:11.416732 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:11.416814 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:11.417354 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:11.417370 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:11.418429 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:11.418434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:11.419700 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:11.419712 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:11.420480 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:11.420487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:11.420709 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:11.421412 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:04:11.421415 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:11.421454 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:04:11.421526 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:11.422047 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:11.422065 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:11.422071 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 01:04:11.422253 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 01:04:11.422614 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":375,"threads_started":1}
I20260809 01:04:11.425277 24586 heartbeater.cc:347] Connected to a master server at 127.4.82.190:46225
I20260809 01:04:11.425571 24586 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:11.426399 24586 heartbeater.cc:511] Master 127.4.82.190:46225 requested a full tablet report, sending...
I20260809 01:04:11.428534 24391 ts_manager.cc:194] Registered new tserver with Master: 625fd4313c28452fb01b54d5915bf09a (127.4.82.129:46849)
I20260809 01:04:11.429970 24391 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.82.129:56929
W20260809 01:04:11.752799 24590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:11.753095 24590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:11.753223 24590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:11.760244 24590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:11.760344 24590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:11.760545 24590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:04:11.769675 24590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46225
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:11.770990 24590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:11.772740 24590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:11.785537 24595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:11.785477 24598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:11.785481 24596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:11.786547 24590 server_base.cc:1034] running on GCE node
I20260809 01:04:11.787143 24590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:11.788107 24590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:11.789340 24590 hybrid_clock.cc:648] HybridClock initialized: now 1786237451789305 us; error 49 us; skew 500 ppm
I20260809 01:04:11.789650 24590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:04:11.792052 24590 webserver.cc:460] Webserver started at http://127.4.82.130:45307/ using document root <none> and password file <none>
I20260809 01:04:11.792788 24590 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:11.792914 24590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:11.793251 24590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:11.796335 24590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "40ff52f9433241b9be1a528efb475014"
format_stamp: "Formatted at 2026-08-09 01:04:11 on dist-test-slave-07c2"
server_key: "3fa2a9a54b02eef6e0f9b7fe2a29deb9"
server_key_iv: "4f702118ae97f0a0d519a27cf258814f"
server_key_version: "encryptionkey@0"
I20260809 01:04:11.797135 24590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "40ff52f9433241b9be1a528efb475014"
format_stamp: "Formatted at 2026-08-09 01:04:11 on dist-test-slave-07c2"
server_key: "3fa2a9a54b02eef6e0f9b7fe2a29deb9"
server_key_iv: "4f702118ae97f0a0d519a27cf258814f"
server_key_version: "encryptionkey@0"
I20260809 01:04:11.802147 24590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 01:04:11.805500 24604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:11.806741 24590 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 01:04:11.806918 24590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "40ff52f9433241b9be1a528efb475014"
format_stamp: "Formatted at 2026-08-09 01:04:11 on dist-test-slave-07c2"
server_key: "3fa2a9a54b02eef6e0f9b7fe2a29deb9"
server_key_iv: "4f702118ae97f0a0d519a27cf258814f"
server_key_version: "encryptionkey@0"
I20260809 01:04:11.807135 24590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:11.821110 24590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:11.821874 24590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:11.822187 24590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:11.823213 24590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:11.824581 24590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:11.824705 24590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:11.824837 24590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:11.824935 24590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:11.858409 24590 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:43067
I20260809 01:04:11.858446 24721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:43067 every 8 connection(s)
I20260809 01:04:11.859892 24590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:04:11.865195 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 24590
I20260809 01:04:11.865381 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:04:11.865867 4426 external_mini_cluster.cc:1428] Setting key 1588838f6128c4dccad39dd40003f493
I20260809 01:04:11.868923 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46225
--builtin_ntp_servers=127.4.82.148:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 01:04:11.874678 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:11.862615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:45365 (local address 127.4.82.190:46225)
0809 01:04:11.862823 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:11.862830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:11.864399 (+ 1569us) server_negotiation.cc:407] Connection header received
0809 01:04:11.866827 (+ 2428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:11.866832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:11.866916 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:11.867027 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:11.868313 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:11.869493 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:11.871754 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:11.872453 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:11.872592 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:11.873890 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:11.873895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:11.873898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:11.873914 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:11.873944 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:11.873955 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:11.874049 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:11.874289 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 01:04:11.874382 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 01:04:11.875823 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:11.863146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:46225 (local address 127.4.82.130:45365)
0809 01:04:11.864229 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:11.864280 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:11.866538 (+ 2258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:11.867267 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:11.867277 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:11.867410 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:11.868132 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:11.868153 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:11.869655 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:11.869660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:11.871603 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:11.871616 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:11.872657 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:11.872662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:11.872861 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:11.873571 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:04:11.873574 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:11.873611 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:04:11.873683 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:11.874050 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:11.874064 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:11.874074 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 01:04:11.874262 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 01:04:11.874623 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":360,"threads_started":1}
I20260809 01:04:11.877178 24722 heartbeater.cc:347] Connected to a master server at 127.4.82.190:46225
I20260809 01:04:11.877473 24722 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:11.878185 24722 heartbeater.cc:511] Master 127.4.82.190:46225 requested a full tablet report, sending...
I20260809 01:04:11.879570 24391 ts_manager.cc:194] Registered new tserver with Master: 40ff52f9433241b9be1a528efb475014 (127.4.82.130:43067)
I20260809 01:04:11.880434 24391 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.82.130:45365
W20260809 01:04:12.201376 24725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:12.201689 24725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:12.201856 24725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:12.208979 24725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:12.209106 24725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:12.209281 24725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:04:12.218184 24725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:46225
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:12.219436 24725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:12.221197 24725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:12.233952 24731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:12.234037 24730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:12.234117 24733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:12.235013 24725 server_base.cc:1034] running on GCE node
I20260809 01:04:12.235636 24725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:12.236467 24725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:12.237666 24725 hybrid_clock.cc:648] HybridClock initialized: now 1786237452237625 us; error 31 us; skew 500 ppm
I20260809 01:04:12.237969 24725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:04:12.240252 24725 webserver.cc:460] Webserver started at http://127.4.82.131:38501/ using document root <none> and password file <none>
I20260809 01:04:12.240922 24725 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:12.241041 24725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:12.241366 24725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:12.244514 24725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "018e83f33e24498298cdf2e46c2d44d4"
format_stamp: "Formatted at 2026-08-09 01:04:12 on dist-test-slave-07c2"
server_key: "9bac9ce722b023e8c3312dc6ef2acb53"
server_key_iv: "ebbdd68b46e1677e4d688debd14f829e"
server_key_version: "encryptionkey@0"
I20260809 01:04:12.245288 24725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "018e83f33e24498298cdf2e46c2d44d4"
format_stamp: "Formatted at 2026-08-09 01:04:12 on dist-test-slave-07c2"
server_key: "9bac9ce722b023e8c3312dc6ef2acb53"
server_key_iv: "ebbdd68b46e1677e4d688debd14f829e"
server_key_version: "encryptionkey@0"
I20260809 01:04:12.250316 24725 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 01:04:12.253552 24740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:12.254789 24725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 01:04:12.254956 24725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "018e83f33e24498298cdf2e46c2d44d4"
format_stamp: "Formatted at 2026-08-09 01:04:12 on dist-test-slave-07c2"
server_key: "9bac9ce722b023e8c3312dc6ef2acb53"
server_key_iv: "ebbdd68b46e1677e4d688debd14f829e"
server_key_version: "encryptionkey@0"
I20260809 01:04:12.255177 24725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:12.277407 24725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:12.278151 24725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:12.278445 24725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:12.279377 24725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:12.280746 24725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:12.280865 24725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:12.280993 24725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:12.281090 24725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:12.313926 24725 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:34505
I20260809 01:04:12.314009 24854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:34505 every 8 connection(s)
I20260809 01:04:12.315228 24725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:04:12.319651 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 24725
I20260809 01:04:12.319828 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:04:12.320300 4426 external_mini_cluster.cc:1428] Setting key b186b6cd089a09c2e91b07ecc500e179
I20260809 01:04:12.327669 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.317621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:43885 (local address 127.4.82.190:46225)
0809 01:04:12.317848 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:12.317855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:12.319174 (+ 1319us) server_negotiation.cc:407] Connection header received
0809 01:04:12.321147 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:12.321152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:12.321233 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:12.321316 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:12.322644 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.323665 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.325145 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.325677 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.325767 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.326976 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:12.326981 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:12.326984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:12.326999 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:12.327028 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:12.327039 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:12.327101 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:12.327275 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 01:04:12.327420 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 01:04:12.328531 24857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.318153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:46225 (local address 127.4.82.131:43885)
0809 01:04:12.319007 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:12.319041 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:12.320881 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:12.321608 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:12.321620 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:12.321720 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:12.322463 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:12.322487 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.323815 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.323818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:12.325001 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:12.325013 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.325862 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.325867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:12.326055 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.326698 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:04:12.326700 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:12.326734 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:04:12.326804 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:12.327115 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:12.327122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:12.327126 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 01:04:12.327250 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 01:04:12.327562 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":283,"threads_started":1}
I20260809 01:04:12.329766 24855 heartbeater.cc:347] Connected to a master server at 127.4.82.190:46225
I20260809 01:04:12.330027 24855 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:12.330650 24855 heartbeater.cc:511] Master 127.4.82.190:46225 requested a full tablet report, sending...
I20260809 01:04:12.331935 24391 ts_manager.cc:194] Registered new tserver with Master: 018e83f33e24498298cdf2e46c2d44d4 (127.4.82.131:34505)
I20260809 01:04:12.332641 24391 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.82.131:43885
I20260809 01:04:12.335426 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:04:12.345042 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.339279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34974 (local address 127.4.82.190:46225)
0809 01:04:12.339467 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:12.339471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:12.339905 (+ 434us) server_negotiation.cc:407] Connection header received
0809 01:04:12.340124 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:12.340128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:12.340192 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:12.340266 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:12.340787 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.341430 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.342683 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.343076 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.343176 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.344358 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:12.344363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:12.344366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:12.344381 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:12.344407 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:12.344418 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:12.344520 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:12.344676 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 01:04:12.344750 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 01:04:12.348625 24391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:04:12.355109 24391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:04:12.378590 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.366987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:43067 (local address 127.0.0.1:60748)
0809 01:04:12.367852 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:12.367886 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:12.368044 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:12.369239 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:12.369246 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:12.369279 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:12.369764 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:12.369784 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.372435 (+ 2651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.372443 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:12.375046 (+ 2603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:12.375074 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.377324 (+ 2250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.377338 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:12.377733 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.377874 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 01:04:12.378004 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 01:04:12.378208 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":221,"thread_start_us":455,"threads_started":1}
I20260809 01:04:12.378634 24870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.368017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52542 (local address 127.4.82.131:34505)
0809 01:04:12.368749 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:12.368756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:12.370249 (+ 1493us) server_negotiation.cc:407] Connection header received
0809 01:04:12.370345 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:12.370351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:12.370567 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:12.370762 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:12.371732 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.373103 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.375337 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.376790 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.376933 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.377195 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:12.378121 (+ 926us) server_negotiation.cc:299] Negotiation successful
0809 01:04:12.378278 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":415,"threads_started":1}
I20260809 01:04:12.379281 24869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.367305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:34505 (local address 127.0.0.1:52542)
0809 01:04:12.369977 (+ 2672us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:12.369995 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:12.370115 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:12.371058 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:12.371063 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:12.371090 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:12.371497 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:12.371514 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.373264 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.373271 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:12.375044 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:12.375060 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.377352 (+ 2292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.377357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:12.377734 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.377874 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 01:04:12.378004 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 01:04:12.378925 (+ 921us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2500,"mutex_wait_us":123,"thread_start_us":2555,"threads_started":1}
I20260809 01:04:12.379702 24868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.367268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.4.82.130:43067)
0809 01:04:12.368648 (+ 1380us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:12.368658 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:12.368688 (+ 30us) server_negotiation.cc:407] Connection header received
0809 01:04:12.368789 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:12.368797 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:12.369020 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:12.369223 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:12.369993 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.372286 (+ 2293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.375437 (+ 3151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.376792 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.376932 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.377178 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:12.379118 (+ 1940us) server_negotiation.cc:299] Negotiation successful
0809 01:04:12.379327 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1192,"thread_start_us":1519,"threads_started":1}
I20260809 01:04:12.378965 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.370034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:46849 (local address 127.0.0.1:44524)
0809 01:04:12.370503 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:12.370537 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:12.370645 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:12.372150 (+ 1505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:12.372158 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:12.372195 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:12.372614 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:12.372625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.374753 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.374760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:12.376561 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:12.376619 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.378077 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.378084 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:12.378264 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.378295 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 01:04:12.378385 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 01:04:12.378509 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":757,"threads_started":1}
I20260809 01:04:12.380064 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.370560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.4.82.129:46849)
0809 01:04:12.371197 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:12.371214 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:12.371242 (+ 28us) server_negotiation.cc:407] Connection header received
0809 01:04:12.371725 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:12.371733 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:12.371934 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:12.372825 (+ 891us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:12.373074 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.374587 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.376840 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.377759 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.378021 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.378283 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:12.378455 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 01:04:12.378609 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":373,"threads_started":1}
I20260809 01:04:12.383381 24653 tablet_service.cc:1467] Processing CreateTablet for tablet 5058bcc4de2c454f9e0f55a32d34562f (DEFAULT_TABLE table=test-table [id=94e8b53e9b594db8b43aee9ced4f292a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:12.383391 24790 tablet_service.cc:1467] Processing CreateTablet for tablet 5058bcc4de2c454f9e0f55a32d34562f (DEFAULT_TABLE table=test-table [id=94e8b53e9b594db8b43aee9ced4f292a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:12.384778 24653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5058bcc4de2c454f9e0f55a32d34562f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:12.384879 24790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5058bcc4de2c454f9e0f55a32d34562f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:12.389048 24516 tablet_service.cc:1467] Processing CreateTablet for tablet 5058bcc4de2c454f9e0f55a32d34562f (DEFAULT_TABLE table=test-table [id=94e8b53e9b594db8b43aee9ced4f292a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:12.389979 24516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5058bcc4de2c454f9e0f55a32d34562f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:12.395563 24874 tablet_bootstrap.cc:492] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4: Bootstrap starting.
I20260809 01:04:12.398625 24874 tablet_bootstrap.cc:654] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:12.399760 24874 log.cc:824] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:12.402218 24876 tablet_bootstrap.cc:492] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014: Bootstrap starting.
I20260809 01:04:12.402599 24874 tablet_bootstrap.cc:492] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4: No bootstrap required, opened a new log
I20260809 01:04:12.402973 24874 ts_tablet_manager.cc:1397] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 01:04:12.404738 24878 tablet_bootstrap.cc:492] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a: Bootstrap starting.
I20260809 01:04:12.405817 24876 tablet_bootstrap.cc:654] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:12.406985 24876 log.cc:824] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:12.407929 24878 tablet_bootstrap.cc:654] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:12.409298 24878 log.cc:824] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:12.415274 24878 tablet_bootstrap.cc:492] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a: No bootstrap required, opened a new log
I20260809 01:04:12.415611 24878 ts_tablet_manager.cc:1397] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.003s
I20260809 01:04:12.416255 24874 raft_consensus.cc:348] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ff52f9433241b9be1a528efb475014" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43067 } } peers { permanent_uuid: "018e83f33e24498298cdf2e46c2d44d4" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34505 } } peers { permanent_uuid: "625fd4313c28452fb01b54d5915bf09a" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46849 } }
I20260809 01:04:12.416916 24874 raft_consensus.cc:374] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:12.417086 24874 raft_consensus.cc:729] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018e83f33e24498298cdf2e46c2d44d4, State: Initialized, Role: FOLLOWER
I20260809 01:04:12.418017 24874 consensus_queue.cc:260] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ff52f9433241b9be1a528efb475014" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43067 } } peers { permanent_uuid: "018e83f33e24498298cdf2e46c2d44d4" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34505 } } peers { permanent_uuid: "625fd4313c28452fb01b54d5915bf09a" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46849 } }
I20260809 01:04:12.419438 24855 heartbeater.cc:503] Master 127.4.82.190:46225 was elected leader, sending a full tablet report...
I20260809 01:04:12.420357 24874 ts_tablet_manager.cc:1428] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
I20260809 01:04:12.423578 24876 tablet_bootstrap.cc:492] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014: No bootstrap required, opened a new log
I20260809 01:04:12.424091 24876 ts_tablet_manager.cc:1397] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.012s
I20260809 01:04:12.427903 24878 raft_consensus.cc:348] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ff52f9433241b9be1a528efb475014" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43067 } } peers { permanent_uuid: "018e83f33e24498298cdf2e46c2d44d4" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34505 } } peers { permanent_uuid: "625fd4313c28452fb01b54d5915bf09a" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46849 } }
I20260809 01:04:12.428341 24878 raft_consensus.cc:374] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:12.428436 24878 raft_consensus.cc:729] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 625fd4313c28452fb01b54d5915bf09a, State: Initialized, Role: FOLLOWER
I20260809 01:04:12.429062 24878 consensus_queue.cc:260] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ff52f9433241b9be1a528efb475014" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43067 } } peers { permanent_uuid: "018e83f33e24498298cdf2e46c2d44d4" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34505 } } peers { permanent_uuid: "625fd4313c28452fb01b54d5915bf09a" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46849 } }
I20260809 01:04:12.430124 24586 heartbeater.cc:503] Master 127.4.82.190:46225 was elected leader, sending a full tablet report...
I20260809 01:04:12.430739 24878 ts_tablet_manager.cc:1428] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260809 01:04:12.433681 24876 raft_consensus.cc:348] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ff52f9433241b9be1a528efb475014" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43067 } } peers { permanent_uuid: "018e83f33e24498298cdf2e46c2d44d4" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34505 } } peers { permanent_uuid: "625fd4313c28452fb01b54d5915bf09a" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46849 } }
I20260809 01:04:12.434072 24876 raft_consensus.cc:374] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:12.434211 24876 raft_consensus.cc:729] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40ff52f9433241b9be1a528efb475014, State: Initialized, Role: FOLLOWER
I20260809 01:04:12.434768 24876 consensus_queue.cc:260] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ff52f9433241b9be1a528efb475014" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43067 } } peers { permanent_uuid: "018e83f33e24498298cdf2e46c2d44d4" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34505 } } peers { permanent_uuid: "625fd4313c28452fb01b54d5915bf09a" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46849 } }
I20260809 01:04:12.435807 24722 heartbeater.cc:503] Master 127.4.82.190:46225 was elected leader, sending a full tablet report...
I20260809 01:04:12.436414 24876 ts_tablet_manager.cc:1428] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014: Time spent starting tablet: real 0.012s user 0.008s sys 0.003s
I20260809 01:04:12.475914 24882 raft_consensus.cc:482] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:04:12.476178 24882 raft_consensus.cc:504] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ff52f9433241b9be1a528efb475014" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43067 } } peers { permanent_uuid: "018e83f33e24498298cdf2e46c2d44d4" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34505 } } peers { permanent_uuid: "625fd4313c28452fb01b54d5915bf09a" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46849 } }
I20260809 01:04:12.477763 24882 leader_election.cc:288] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40ff52f9433241b9be1a528efb475014 (127.4.82.130:43067), 018e83f33e24498298cdf2e46c2d44d4 (127.4.82.131:34505)
I20260809 01:04:12.486506 24870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.478309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:48875 (local address 127.4.82.131:34505)
0809 01:04:12.478480 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:12.478486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:12.479692 (+ 1206us) server_negotiation.cc:407] Connection header received
0809 01:04:12.479831 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:12.479835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:12.479889 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:12.479956 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:12.480499 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.481452 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.484123 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.485375 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.485536 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.485603 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:12.486140 (+ 537us) server_negotiation.cc:299] Negotiation successful
0809 01:04:12.486261 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 01:04:12.486506 24868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.478304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:60469 (local address 127.4.82.130:43067)
0809 01:04:12.478480 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:12.478486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:12.479027 (+ 541us) server_negotiation.cc:407] Connection header received
0809 01:04:12.479320 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:12.479325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:12.479433 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:12.479532 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:12.480202 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.481190 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.484163 (+ 2973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.485369 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.485539 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.485603 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:12.486140 (+ 537us) server_negotiation.cc:299] Negotiation successful
0809 01:04:12.486266 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 01:04:12.486668 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.478219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:43067 (local address 127.4.82.129:60469)
0809 01:04:12.478898 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:12.478913 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:12.479088 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:12.479581 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:12.479586 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:12.479620 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:12.480037 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:12.480047 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.481404 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.481408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:12.483903 (+ 2495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:12.483919 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.485688 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.485698 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:12.485923 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.485996 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 01:04:12.486095 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 01:04:12.486247 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":42,"thread_start_us":986,"threads_started":1}
I20260809 01:04:12.486793 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.478237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:34505 (local address 127.4.82.129:48875)
0809 01:04:12.479566 (+ 1329us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:12.479583 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:12.479705 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:12.480020 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:12.480025 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:12.480047 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:12.480353 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:12.480361 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.481626 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.481638 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:12.483910 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:12.483919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.485725 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.485733 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:12.485961 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.486010 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 01:04:12.486095 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 01:04:12.486247 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1191,"mutex_wait_us":159,"thread_start_us":391,"threads_started":1}
I20260809 01:04:12.487743 24675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5058bcc4de2c454f9e0f55a32d34562f" candidate_uuid: "625fd4313c28452fb01b54d5915bf09a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40ff52f9433241b9be1a528efb475014" is_pre_election: true
I20260809 01:04:12.487850 24810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5058bcc4de2c454f9e0f55a32d34562f" candidate_uuid: "625fd4313c28452fb01b54d5915bf09a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018e83f33e24498298cdf2e46c2d44d4" is_pre_election: true
I20260809 01:04:12.488256 24675 raft_consensus.cc:2454] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 625fd4313c28452fb01b54d5915bf09a in term 0.
I20260809 01:04:12.488358 24810 raft_consensus.cc:2454] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 625fd4313c28452fb01b54d5915bf09a in term 0.
I20260809 01:04:12.489037 24470 leader_election.cc:302] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40ff52f9433241b9be1a528efb475014, 625fd4313c28452fb01b54d5915bf09a; no voters:
I20260809 01:04:12.489429 24882 raft_consensus.cc:2781] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:04:12.489570 24882 raft_consensus.cc:482] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:04:12.489686 24882 raft_consensus.cc:3037] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:12.492117 24882 raft_consensus.cc:504] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ff52f9433241b9be1a528efb475014" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43067 } } peers { permanent_uuid: "018e83f33e24498298cdf2e46c2d44d4" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34505 } } peers { permanent_uuid: "625fd4313c28452fb01b54d5915bf09a" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46849 } }
I20260809 01:04:12.492727 24882 leader_election.cc:288] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [CANDIDATE]: Term 1 election: Requested vote from peers 40ff52f9433241b9be1a528efb475014 (127.4.82.130:43067), 018e83f33e24498298cdf2e46c2d44d4 (127.4.82.131:34505)
I20260809 01:04:12.493156 24675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5058bcc4de2c454f9e0f55a32d34562f" candidate_uuid: "625fd4313c28452fb01b54d5915bf09a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40ff52f9433241b9be1a528efb475014"
I20260809 01:04:12.493233 24810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5058bcc4de2c454f9e0f55a32d34562f" candidate_uuid: "625fd4313c28452fb01b54d5915bf09a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "018e83f33e24498298cdf2e46c2d44d4"
I20260809 01:04:12.493371 24675 raft_consensus.cc:3037] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:12.493402 24810 raft_consensus.cc:3037] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:12.496696 24810 raft_consensus.cc:2454] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 625fd4313c28452fb01b54d5915bf09a in term 1.
I20260809 01:04:12.496717 24675 raft_consensus.cc:2454] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 625fd4313c28452fb01b54d5915bf09a in term 1.
I20260809 01:04:12.497154 24469 leader_election.cc:302] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 018e83f33e24498298cdf2e46c2d44d4, 625fd4313c28452fb01b54d5915bf09a; no voters:
I20260809 01:04:12.497471 24882 raft_consensus.cc:2781] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:12.497925 24882 raft_consensus.cc:686] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [term 1 LEADER]: Becoming Leader. State: Replica: 625fd4313c28452fb01b54d5915bf09a, State: Running, Role: LEADER
I20260809 01:04:12.498381 24882 consensus_queue.cc:237] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ff52f9433241b9be1a528efb475014" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43067 } } peers { permanent_uuid: "018e83f33e24498298cdf2e46c2d44d4" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34505 } } peers { permanent_uuid: "625fd4313c28452fb01b54d5915bf09a" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46849 } }
I20260809 01:04:12.502547 24391 catalog_manager.cc:5495] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a reported cstate change: term changed from 0 to 1, leader changed from <none> to 625fd4313c28452fb01b54d5915bf09a (127.4.82.129). New cstate: current_term: 1 leader_uuid: "625fd4313c28452fb01b54d5915bf09a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40ff52f9433241b9be1a528efb475014" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "018e83f33e24498298cdf2e46c2d44d4" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "625fd4313c28452fb01b54d5915bf09a" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 46849 } health_report { overall_health: HEALTHY } } }
I20260809 01:04:12.533215 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.528187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44534 (local address 127.4.82.129:46849)
0809 01:04:12.528377 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:12.528381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:12.528496 (+ 115us) server_negotiation.cc:407] Connection header received
0809 01:04:12.528669 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:12.528673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:12.528760 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:12.528887 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:04:12.529227 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.529912 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.531275 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.531699 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.531790 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.532261 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:04:12.532710 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:04:12.532840 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:12.532971 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 01:04:12.533006 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 01:04:12.541500 24675 raft_consensus.cc:1261] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014 [term 1 FOLLOWER]: Refusing update from remote peer 625fd4313c28452fb01b54d5915bf09a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:04:12.541501 24810 raft_consensus.cc:1261] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4 [term 1 FOLLOWER]: Refusing update from remote peer 625fd4313c28452fb01b54d5915bf09a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:04:12.542551 24887 consensus_queue.cc:1035] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [LEADER]: Connected to new peer: Peer: permanent_uuid: "40ff52f9433241b9be1a528efb475014" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 43067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:04:12.542929 24882 consensus_queue.cc:1035] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [LEADER]: Connected to new peer: Peer: permanent_uuid: "018e83f33e24498298cdf2e46c2d44d4" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 34505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:04:12.568912 24391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:04:12.569156 24391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:04:12.572775 24391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc4d59a3a0a74c79a55b474f45180cb1: Sending DeleteTablet for 3 replicas of tablet 5058bcc4de2c454f9e0f55a32d34562f
I20260809 01:04:12.573930 24790 tablet_service.cc:1514] Processing DeleteTablet for tablet 5058bcc4de2c454f9e0f55a32d34562f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:12 UTC) from {username='slave'} at 127.0.0.1:52542
I20260809 01:04:12.574000 24653 tablet_service.cc:1514] Processing DeleteTablet for tablet 5058bcc4de2c454f9e0f55a32d34562f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:12 UTC) from {username='slave'} at 127.0.0.1:60748
I20260809 01:04:12.575839 24516 tablet_service.cc:1514] Processing DeleteTablet for tablet 5058bcc4de2c454f9e0f55a32d34562f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:12 UTC) from {username='slave'} at 127.0.0.1:44524
I20260809 01:04:12.576630 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 24454
I20260809 01:04:12.586263 24899 tablet_replica.cc:330] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4: stopping tablet replica
I20260809 01:04:12.586818 24899 raft_consensus.cc:2229] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:04:12.587440 24899 raft_consensus.cc:2258] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:12.589421 24899 ts_tablet_manager.cc:1905] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:12.597363 24899 ts_tablet_manager.cc:1918] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:12.597579 24899 log.cc:1196] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/5058bcc4de2c454f9e0f55a32d34562f
I20260809 01:04:12.598168 24899 ts_tablet_manager.cc:1939] T 5058bcc4de2c454f9e0f55a32d34562f P 018e83f33e24498298cdf2e46c2d44d4: Deleting consensus metadata
I20260809 01:04:12.600044 24375 catalog_manager.cc:4841] TS 018e83f33e24498298cdf2e46c2d44d4 (127.4.82.131:34505): tablet 5058bcc4de2c454f9e0f55a32d34562f (table test-table [id=94e8b53e9b594db8b43aee9ced4f292a]) successfully deleted
I20260809 01:04:12.604074 24901 tablet_replica.cc:330] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a: stopping tablet replica
I20260809 01:04:12.604600 24901 raft_consensus.cc:2229] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:04:12.605405 24901 raft_consensus.cc:2258] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:12.606297 24900 tablet_replica.cc:330] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014: stopping tablet replica
I20260809 01:04:12.606724 24900 raft_consensus.cc:2229] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:04:12.607255 24900 raft_consensus.cc:2258] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:12.608042 24901 ts_tablet_manager.cc:1905] T 5058bcc4de2c454f9e0f55a32d34562f P 625fd4313c28452fb01b54d5915bf09a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:12.609372 24900 ts_tablet_manager.cc:1905] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:12.612489 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.581618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44540 (local address 127.4.82.129:46849)
0809 01:04:12.581844 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:12.581853 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:12.603418 (+ 21565us) server_negotiation.cc:407] Connection header received
0809 01:04:12.603797 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:12.603806 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:12.603973 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:12.604063 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:12.604902 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.606302 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.608203 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.608886 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.608990 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.610545 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:12.610571 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:12.610586 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:12.611402 (+ 816us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:12.611454 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:12.611484 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:12.611587 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:12.611975 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 01:04:12.612080 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 01:04:12.613824 24580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:12.616206 24900 ts_tablet_manager.cc:1918] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:12.616431 24900 log.cc:1196] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/5058bcc4de2c454f9e0f55a32d34562f
I20260809 01:04:12.616986 24900 ts_tablet_manager.cc:1939] T 5058bcc4de2c454f9e0f55a32d34562f P 40ff52f9433241b9be1a528efb475014: Deleting consensus metadata
I20260809 01:04:12.618810 24376 catalog_manager.cc:4841] TS 40ff52f9433241b9be1a528efb475014 (127.4.82.130:43067): tablet 5058bcc4de2c454f9e0f55a32d34562f (table test-table [id=94e8b53e9b594db8b43aee9ced4f292a]) successfully deleted
I20260809 01:04:12.712635 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 24454
I20260809 01:04:12.733191 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 24590
W20260809 01:04:12.733269 24376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.82.129:46849 (error 108)
I20260809 01:04:12.735170 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.734723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:46849 (local address 127.0.0.1:44550)
0809 01:04:12.734918 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:12.735031 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 01:04:12.735432 24376 catalog_manager.cc:4584] TS 625fd4313c28452fb01b54d5915bf09a (127.4.82.129:46849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5058bcc4de2c454f9e0f55a32d34562f: Network error: Client connection negotiation failed: client connection to 127.4.82.129:46849: connect: Connection refused (error 111)
I20260809 01:04:12.740206 24868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.734055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60756 (local address 127.4.82.130:43067)
0809 01:04:12.734262 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:12.734270 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:12.734296 (+ 26us) server_negotiation.cc:407] Connection header received
0809 01:04:12.734446 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:12.734452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:12.734558 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:12.734665 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:12.735095 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.735990 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.737277 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.737707 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.737846 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.738848 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:12.738857 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:12.738870 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:12.739482 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:12.739526 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:12.739546 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:12.739669 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:12.739805 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 01:04:12.739904 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 01:04:12.740859 24716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:12.788960 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.788442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:46849 (local address 127.0.0.1:44558)
0809 01:04:12.788710 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:12.788798 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 01:04:12.790030 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.789682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:46849 (local address 127.0.0.1:44570)
0809 01:04:12.789841 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:12.789913 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 01:04:12.829617 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 24590
I20260809 01:04:12.848784 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 24725
I20260809 01:04:12.850514 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.850098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:46849 (local address 127.0.0.1:44584)
0809 01:04:12.850280 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:12.850354 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 01:04:12.851590 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.851185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:46849 (local address 127.0.0.1:44588)
0809 01:04:12.851369 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:12.851451 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 01:04:12.856122 24870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.849559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.4.82.131:34505)
0809 01:04:12.849748 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:12.849752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:12.849792 (+ 40us) server_negotiation.cc:407] Connection header received
0809 01:04:12.850019 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:12.850022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:12.850108 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:12.850223 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:12.850674 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.851734 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.853047 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:12.853508 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:12.853646 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:12.854741 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:12.854752 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:12.854766 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:12.855390 (+ 624us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:12.855460 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:12.855487 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:12.855610 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:12.855721 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 01:04:12.855824 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 01:04:12.856767 24850 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:12.932243 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.931775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:46849 (local address 127.0.0.1:44590)
0809 01:04:12.931988 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:12.932074 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 01:04:12.933300 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:12.932901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:46849 (local address 127.0.0.1:44604)
0809 01:04:12.933110 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:12.933185 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:46849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260809 01:04:12.943739 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 24725
I20260809 01:04:12.962421 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 24360
I20260809 01:04:12.963551 24422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:13.070209 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 24360
2026-08-09T01:04:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2740 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:13 dist-test-slave-07c2 krb5kdc[24912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:13 dist-test-slave-07c2 krb5kdc[24912](info): set up 2 sockets
Aug 09 01:04:13 dist-test-slave-07c2 krb5kdc[24912](info): commencing operation
krb5kdc: starting...
W20260809 01:04:14.231514 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:14 dist-test-slave-07c2 krb5kdc[24912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:04:14.514950 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34911
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:34911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:14.852353 24925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:14.852624 24925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:14.852867 24925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:14.859897 24925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:14.859998 24925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:14.860109 24925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:14.860224 24925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:14.860306 24925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:14.869831 24925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:34911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:14.871078 24925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:14.872351 24925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:14.884358 24933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:14.884405 24931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:14.884445 24930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:14.885528 24925 server_base.cc:1034] running on GCE node
I20260809 01:04:14.886309 24925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:14.887748 24925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:14.888952 24925 hybrid_clock.cc:648] HybridClock initialized: now 1786237454888899 us; error 38 us; skew 500 ppm
I20260809 01:04:14.889302 24925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:14 dist-test-slave-07c2 krb5kdc[24912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237454, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:14.898408 24925 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:04:14.900027 24925 webserver.cc:460] Webserver started at http://127.4.82.190:45653/ using document root <none> and password file <none>
I20260809 01:04:14.900678 24925 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:14.900794 24925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:14.901032 24925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:14.904090 24925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "1987cb9a7b8d41159d8a653bbb737398"
format_stamp: "Formatted at 2026-08-09 01:04:14 on dist-test-slave-07c2"
server_key: "31fdeb1b454c5e50d0dd5fc244466cba"
server_key_iv: "413914e79d71cbe5869e98297175b8fc"
server_key_version: "encryptionkey@0"
I20260809 01:04:14.904800 24925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "1987cb9a7b8d41159d8a653bbb737398"
format_stamp: "Formatted at 2026-08-09 01:04:14 on dist-test-slave-07c2"
server_key: "31fdeb1b454c5e50d0dd5fc244466cba"
server_key_iv: "413914e79d71cbe5869e98297175b8fc"
server_key_version: "encryptionkey@0"
I20260809 01:04:14.909821 24925 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 01:04:14.913154 24940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:14.914446 24925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 01:04:14.914582 24925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "1987cb9a7b8d41159d8a653bbb737398"
format_stamp: "Formatted at 2026-08-09 01:04:14 on dist-test-slave-07c2"
server_key: "31fdeb1b454c5e50d0dd5fc244466cba"
server_key_iv: "413914e79d71cbe5869e98297175b8fc"
server_key_version: "encryptionkey@0"
I20260809 01:04:14.914744 24925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:14.932367 24925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:14.936198 24925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:14.936430 24925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:14.954967 24925 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:34911
I20260809 01:04:14.955034 24993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:34911 every 8 connection(s)
I20260809 01:04:14.956321 24925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:14.959813 24994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:14.964359 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 24925
I20260809 01:04:14.964530 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:14.964970 4426 external_mini_cluster.cc:1428] Setting key 1bd7c1316f66747afaf775e86e6c4690
I20260809 01:04:14.970314 24994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398: Bootstrap starting.
I20260809 01:04:14.973932 24994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:14.975188 24994 log.cc:824] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:14.978590 24994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398: No bootstrap required, opened a new log
Aug 09 01:04:14 dist-test-slave-07c2 krb5kdc[24912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:14.986351 24994 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1987cb9a7b8d41159d8a653bbb737398" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34911 } }
I20260809 01:04:14.986713 24994 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:14.986847 24994 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1987cb9a7b8d41159d8a653bbb737398, State: Initialized, Role: FOLLOWER
I20260809 01:04:14.987540 24994 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1987cb9a7b8d41159d8a653bbb737398" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34911 } }
I20260809 01:04:14.987843 24994 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:14.987993 24994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:14.988196 24994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:14.989099 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:14.966759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42344 (local address 127.4.82.190:34911)
0809 01:04:14.967679 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:14.967690 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:14.967738 (+ 48us) server_negotiation.cc:407] Connection header received
0809 01:04:14.969459 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:14.969480 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:14.969833 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:14.970125 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:14.972137 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:14.973386 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:14.975390 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:14.976445 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:14.976616 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:14.981390 (+ 4774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:14.981401 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:14.981413 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:14.981451 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:14.984618 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:14.985319 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:14.985326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:14.985331 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:14.985491 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:14.985901 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:14.985904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:14.985907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:14.986415 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:14.986712 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:14.987223 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 01:04:14.987680 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":410,"threads_started":1}
I20260809 01:04:14.991253 24994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1987cb9a7b8d41159d8a653bbb737398" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34911 } }
I20260809 01:04:14.991886 24994 leader_election.cc:302] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1987cb9a7b8d41159d8a653bbb737398; no voters:
I20260809 01:04:14.992461 24994 leader_election.cc:288] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:14.992595 24999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:14.993332 24999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [term 1 LEADER]: Becoming Leader. State: Replica: 1987cb9a7b8d41159d8a653bbb737398, State: Running, Role: LEADER
I20260809 01:04:14.993884 24999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1987cb9a7b8d41159d8a653bbb737398" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34911 } }
I20260809 01:04:14.994465 24994 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:14.996714 25001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1987cb9a7b8d41159d8a653bbb737398. Latest consensus state: current_term: 1 leader_uuid: "1987cb9a7b8d41159d8a653bbb737398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1987cb9a7b8d41159d8a653bbb737398" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34911 } } }
I20260809 01:04:14.997170 25001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:14.999571 25000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1987cb9a7b8d41159d8a653bbb737398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1987cb9a7b8d41159d8a653bbb737398" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34911 } } }
I20260809 01:04:14.999939 25000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:15.000504 25008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:15.005155 25008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:15.013248 25008 catalog_manager.cc:1347] Generated new cluster ID: 768e6f6730994f56b5190ebb03cc92ac
I20260809 01:04:15.013374 25008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:15.024123 25008 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:15.025977 25008 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:15.036237 25008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398: Generated new TSK 0
I20260809 01:04:15.037067 25008 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.82.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
Entry for principal kudu/127.4.82.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab.
I20260809 01:04:15.104709 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34911
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:15.438684 25021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:15.438951 25021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:15.439046 25021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:15.446310 25021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:15.446472 25021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:15.446683 25021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:04:15.456329 25021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.129.keytab
--principal=kudu/127.4.82.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:15.457571 25021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:15.458864 25021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:15.471882 25027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:15.471969 25028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:15.471935 25030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:15.472672 25021 server_base.cc:1034] running on GCE node
I20260809 01:04:15.473266 25021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:15.474114 25021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:15.475371 25021 hybrid_clock.cc:648] HybridClock initialized: now 1786237455475328 us; error 34 us; skew 500 ppm
I20260809 01:04:15.475668 25021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:15 dist-test-slave-07c2 krb5kdc[24912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237455, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:15.478816 25021 init.cc:377] Logged in from keytab as kudu/127.4.82.129@KRBTEST.COM (short username kudu)
I20260809 01:04:15.480271 25021 webserver.cc:460] Webserver started at http://127.4.82.129:44583/ using document root <none> and password file <none>
I20260809 01:04:15.480916 25021 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:15.481020 25021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:15.481279 25021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:15.484122 25021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "6f03be45d9814ddd91ec96587175deff"
format_stamp: "Formatted at 2026-08-09 01:04:15 on dist-test-slave-07c2"
server_key: "b5e06da16102376c59797969a3e478c6"
server_key_iv: "b03ecb02d76698b71655d1c16f9bf08b"
server_key_version: "encryptionkey@0"
I20260809 01:04:15.484844 25021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "6f03be45d9814ddd91ec96587175deff"
format_stamp: "Formatted at 2026-08-09 01:04:15 on dist-test-slave-07c2"
server_key: "b5e06da16102376c59797969a3e478c6"
server_key_iv: "b03ecb02d76698b71655d1c16f9bf08b"
server_key_version: "encryptionkey@0"
I20260809 01:04:15.489843 25021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 01:04:15.493440 25038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:15.494860 25021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 01:04:15.495042 25021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "6f03be45d9814ddd91ec96587175deff"
format_stamp: "Formatted at 2026-08-09 01:04:15 on dist-test-slave-07c2"
server_key: "b5e06da16102376c59797969a3e478c6"
server_key_iv: "b03ecb02d76698b71655d1c16f9bf08b"
server_key_version: "encryptionkey@0"
I20260809 01:04:15.495232 25021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:15.539050 25021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:15.543181 25021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:15.543515 25021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:15.544481 25021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:15.545809 25021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:15.545885 25021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:15.546063 25021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:15.546123 25021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:15.584722 25021 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:45373
I20260809 01:04:15.584774 25153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:45373 every 8 connection(s)
I20260809 01:04:15.586097 25021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:04:15.592803 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 25021
I20260809 01:04:15.593011 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:04:15.593467 4426 external_mini_cluster.cc:1428] Setting key 9fca478b4b281d467353534389ce52ec
Aug 09 01:04:15 dist-test-slave-07c2 krb5kdc[24912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237455, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.129@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:15.608884 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:15.588726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:52445 (local address 127.4.82.190:34911)
0809 01:04:15.589210 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:15.589216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:15.590036 (+ 820us) server_negotiation.cc:407] Connection header received
0809 01:04:15.591955 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:15.591961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:15.592061 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:15.592181 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:15.593769 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:15.594821 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:15.597244 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:15.597631 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:15.597740 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:15.601991 (+ 4251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:15.601999 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:15.602007 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:15.602051 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:15.605580 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:15.606284 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:15.606292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:15.606297 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:15.606441 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:15.606839 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:15.606846 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:15.606853 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:15.607235 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:15.607421 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:15.608093 (+ 672us) server_negotiation.cc:299] Negotiation successful
0809 01:04:15.608363 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":258,"threads_started":1}
I20260809 01:04:15.609531 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:15.589190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34911 (local address 127.4.82.129:52445)
0809 01:04:15.589880 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:15.589917 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:15.591678 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:15.592411 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:15.592437 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:15.592928 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:15.593580 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:15.593605 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:15.594987 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:15.594991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:15.597087 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:15.597106 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:15.597863 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:15.597873 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:15.598226 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:15.598965 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:04:15.598998 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:15.601776 (+ 2778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:15.605750 (+ 3974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:15.605756 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:15.605766 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:15.606156 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:15.606543 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:15.606546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:15.606548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:15.606725 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:15.607397 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:15.607402 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:15.607684 (+ 282us) client_negotiation.cc:769] Sending connection context
0809 01:04:15.607924 (+ 240us) client_negotiation.cc:240] Negotiation successful
0809 01:04:15.608301 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":292,"threads_started":1}
I20260809 01:04:15.611047 25154 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34911
I20260809 01:04:15.611380 25154 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:15.612205 25154 heartbeater.cc:511] Master 127.4.82.190:34911 requested a full tablet report, sending...
I20260809 01:04:15.614305 24957 ts_manager.cc:194] Registered new tserver with Master: 6f03be45d9814ddd91ec96587175deff (127.4.82.129:45373)
I20260809 01:04:15.615805 24957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.129@KRBTEST.COM'} at 127.4.82.129:52445
WARNING: no policy specified for kudu/127.4.82.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
Entry for principal kudu/127.4.82.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab.
I20260809 01:04:15.658964 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34911
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:15.979013 25160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:15.979419 25160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:15.979602 25160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:15.986763 25160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:15.986897 25160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:15.987087 25160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:04:15.997462 25160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.130.keytab
--principal=kudu/127.4.82.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:15.998703 25160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:16.000105 25160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:16.012543 25166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:16.012655 25168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:16.012512 25170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:16.013218 25160 server_base.cc:1034] running on GCE node
I20260809 01:04:16.013824 25160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:16.014719 25160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:16.015919 25160 hybrid_clock.cc:648] HybridClock initialized: now 1786237456015869 us; error 44 us; skew 500 ppm
I20260809 01:04:16.016185 25160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:16 dist-test-slave-07c2 krb5kdc[24912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237456, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:16.019364 25160 init.cc:377] Logged in from keytab as kudu/127.4.82.130@KRBTEST.COM (short username kudu)
I20260809 01:04:16.020655 25160 webserver.cc:460] Webserver started at http://127.4.82.130:45527/ using document root <none> and password file <none>
I20260809 01:04:16.021296 25160 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:16.021427 25160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:16.021742 25160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:16.024580 25160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "9d486a08f2984a0b8b755f5d67547205"
format_stamp: "Formatted at 2026-08-09 01:04:16 on dist-test-slave-07c2"
server_key: "232db18dc79c4bd166f58f0361f2b036"
server_key_iv: "6cd7b40fb838c4df61c7460290a44acb"
server_key_version: "encryptionkey@0"
I20260809 01:04:16.025384 25160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "9d486a08f2984a0b8b755f5d67547205"
format_stamp: "Formatted at 2026-08-09 01:04:16 on dist-test-slave-07c2"
server_key: "232db18dc79c4bd166f58f0361f2b036"
server_key_iv: "6cd7b40fb838c4df61c7460290a44acb"
server_key_version: "encryptionkey@0"
I20260809 01:04:16.030372 25160 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260809 01:04:16.033779 25177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:16.034999 25160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 01:04:16.035176 25160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "9d486a08f2984a0b8b755f5d67547205"
format_stamp: "Formatted at 2026-08-09 01:04:16 on dist-test-slave-07c2"
server_key: "232db18dc79c4bd166f58f0361f2b036"
server_key_iv: "6cd7b40fb838c4df61c7460290a44acb"
server_key_version: "encryptionkey@0"
I20260809 01:04:16.035418 25160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:16.053335 25160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:16.057144 25160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:16.057444 25160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:16.058344 25160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:16.059629 25160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:16.059705 25160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:16.059882 25160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:16.059934 25160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:16.092868 25160 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:41865
I20260809 01:04:16.092935 25291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:41865 every 8 connection(s)
I20260809 01:04:16.094063 25160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:04:16.095093 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 25160
I20260809 01:04:16.095325 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:04:16.095904 4426 external_mini_cluster.cc:1428] Setting key 09079ba7edb661fb4cdfa5294bd89a1c
Aug 09 01:04:16 dist-test-slave-07c2 krb5kdc[24912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237456, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.130@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:16.114907 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:16.096824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:49035 (local address 127.4.82.190:34911)
0809 01:04:16.097033 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:16.097037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:16.098473 (+ 1436us) server_negotiation.cc:407] Connection header received
0809 01:04:16.100636 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:16.100642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:16.100734 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:16.100841 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:16.102335 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.103039 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.104607 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.105025 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.105162 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:16.108578 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:16.108583 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:16.108586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:16.108616 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:16.111720 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:16.112629 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:16.112633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:16.112635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:16.112777 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:16.113271 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:16.113275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:16.113277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:16.113536 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:16.113650 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:16.114490 (+ 840us) server_negotiation.cc:299] Negotiation successful
0809 01:04:16.114656 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 01:04:16.116354 25294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:16.097344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34911 (local address 127.4.82.130:49035)
0809 01:04:16.098289 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:16.098364 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:16.100389 (+ 2025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:16.101080 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:16.101089 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:16.101620 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:16.102173 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:16.102190 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.103188 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.103194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:16.104455 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:16.104467 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.105250 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.105257 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:16.105489 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:16.106145 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:04:16.106171 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:16.108360 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:16.111870 (+ 3510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:16.111881 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:16.111898 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:16.112495 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:16.112890 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:16.112897 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:16.112904 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:16.113133 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:16.113662 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:16.113672 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:16.114088 (+ 416us) client_negotiation.cc:769] Sending connection context
0809 01:04:16.114369 (+ 281us) client_negotiation.cc:240] Negotiation successful
0809 01:04:16.114951 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":487,"threads_started":1}
I20260809 01:04:16.117790 25292 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34911
I20260809 01:04:16.118119 25292 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:16.118954 25292 heartbeater.cc:511] Master 127.4.82.190:34911 requested a full tablet report, sending...
I20260809 01:04:16.120468 24957 ts_manager.cc:194] Registered new tserver with Master: 9d486a08f2984a0b8b755f5d67547205 (127.4.82.130:41865)
I20260809 01:04:16.121254 24957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.130@KRBTEST.COM'} at 127.4.82.130:49035
WARNING: no policy specified for kudu/127.4.82.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
Entry for principal kudu/127.4.82.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab.
I20260809 01:04:16.162475 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34911
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:16.488894 25297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:16.489179 25297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:16.489351 25297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:16.496315 25297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:16.496444 25297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:16.496601 25297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:04:16.505861 25297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.131.keytab
--principal=kudu/127.4.82.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:16.507006 25297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:16.508283 25297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:16.521199 25305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:16.521273 25302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:16.521277 25303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:16.522102 25297 server_base.cc:1034] running on GCE node
I20260809 01:04:16.522667 25297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:16.523551 25297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:16.524741 25297 hybrid_clock.cc:648] HybridClock initialized: now 1786237456524714 us; error 29 us; skew 500 ppm
I20260809 01:04:16.525028 25297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:16 dist-test-slave-07c2 krb5kdc[24912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237456, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:16.528010 25297 init.cc:377] Logged in from keytab as kudu/127.4.82.131@KRBTEST.COM (short username kudu)
I20260809 01:04:16.529474 25297 webserver.cc:460] Webserver started at http://127.4.82.131:38399/ using document root <none> and password file <none>
I20260809 01:04:16.530100 25297 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:16.530253 25297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:16.530566 25297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:16.533538 25297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "1ef514f7194d4355bc6706e207863b5c"
format_stamp: "Formatted at 2026-08-09 01:04:16 on dist-test-slave-07c2"
server_key: "56cc4625a0d887245ad08473e119387a"
server_key_iv: "043037b84d5d33d0a31be6d7d6c4d6d2"
server_key_version: "encryptionkey@0"
I20260809 01:04:16.534322 25297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "1ef514f7194d4355bc6706e207863b5c"
format_stamp: "Formatted at 2026-08-09 01:04:16 on dist-test-slave-07c2"
server_key: "56cc4625a0d887245ad08473e119387a"
server_key_iv: "043037b84d5d33d0a31be6d7d6c4d6d2"
server_key_version: "encryptionkey@0"
I20260809 01:04:16.539256 25297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 01:04:16.542512 25312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:16.543782 25297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 01:04:16.543943 25297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "1ef514f7194d4355bc6706e207863b5c"
format_stamp: "Formatted at 2026-08-09 01:04:16 on dist-test-slave-07c2"
server_key: "56cc4625a0d887245ad08473e119387a"
server_key_iv: "043037b84d5d33d0a31be6d7d6c4d6d2"
server_key_version: "encryptionkey@0"
I20260809 01:04:16.544145 25297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:16.576327 25297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:16.580255 25297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:16.580580 25297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:16.581570 25297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:16.582907 25297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:16.582980 25297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:16.583170 25297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:16.583225 25297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:16.617615 25297 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:46607
I20260809 01:04:16.617707 25429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:46607 every 8 connection(s)
I20260809 01:04:16.619011 25297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:04:16.619304 25154 heartbeater.cc:503] Master 127.4.82.190:34911 was elected leader, sending a full tablet report...
I20260809 01:04:16.621392 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 25297
I20260809 01:04:16.621595 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:04:16.622206 4426 external_mini_cluster.cc:1428] Setting key 7ce66c0f8af2ad0e70faae59cb331250
Aug 09 01:04:16 dist-test-slave-07c2 krb5kdc[24912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237456, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.131@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:16.639225 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:16.621699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:47695 (local address 127.4.82.190:34911)
0809 01:04:16.622180 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:16.622185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:16.623627 (+ 1442us) server_negotiation.cc:407] Connection header received
0809 01:04:16.625748 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:16.625752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:16.625812 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:16.625913 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:16.627233 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.628064 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.629517 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.629904 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.629994 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:16.633348 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:16.633353 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:16.633356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:16.633385 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:16.636195 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:16.636860 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:16.636868 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:16.636874 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:16.637003 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:16.637432 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:16.637437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:16.637440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:16.637706 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:16.637865 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:16.638602 (+ 737us) server_negotiation.cc:299] Negotiation successful
0809 01:04:16.638823 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":248,"threads_started":1}
I20260809 01:04:16.640055 25433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:16.622388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34911 (local address 127.4.82.131:47695)
0809 01:04:16.623444 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:16.623489 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:16.625512 (+ 2023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:16.626107 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:16.626113 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:16.626598 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:16.627094 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:16.627111 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.628201 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.628204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:16.629393 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:16.629404 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.630083 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.630088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:16.630264 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:16.630920 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:04:16.630945 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:16.633172 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:16.636311 (+ 3139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:16.636317 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:16.636327 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:16.636732 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:16.637127 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:16.637132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:16.637137 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:16.637307 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:16.637833 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:16.637841 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:16.638184 (+ 343us) client_negotiation.cc:769] Sending connection context
0809 01:04:16.638494 (+ 310us) client_negotiation.cc:240] Negotiation successful
0809 01:04:16.639002 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"thread_start_us":602,"threads_started":1}
I20260809 01:04:16.641249 25430 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34911
I20260809 01:04:16.641481 25430 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:16.642096 25430 heartbeater.cc:511] Master 127.4.82.190:34911 requested a full tablet report, sending...
I20260809 01:04:16.643378 24957 ts_manager.cc:194] Registered new tserver with Master: 1ef514f7194d4355bc6706e207863b5c (127.4.82.131:46607)
I20260809 01:04:16.644160 24957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.82.131@KRBTEST.COM'} at 127.4.82.131:47695
I20260809 01:04:16.649235 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:04:16.664373 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:16.653179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.4.82.190:34911)
0809 01:04:16.653328 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:16.653333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:16.653779 (+ 446us) server_negotiation.cc:407] Connection header received
0809 01:04:16.653916 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:16.653919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:16.653972 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:16.654047 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:16.655390 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.656052 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.657282 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.657651 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.657742 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:16.659889 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:16.659893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:16.659896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:16.659923 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:16.662184 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:16.662760 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:16.662763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:16.662765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:16.662871 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:16.663187 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:16.663190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:16.663192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:16.663543 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:16.663733 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:16.664004 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 01:04:16.664157 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 01:04:16.668176 24957 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:04:16.674922 24957 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:04:16.708395 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:16.686435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51572 (local address 127.4.82.130:41865)
0809 01:04:16.700840 (+ 14405us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:16.700848 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:16.700872 (+ 24us) server_negotiation.cc:407] Connection header received
0809 01:04:16.700949 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:16.700955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:16.701195 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:16.701353 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:16.702416 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.703727 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.705933 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.707099 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.707242 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:16.707481 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:16.707853 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 01:04:16.708024 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14181,"thread_start_us":14083,"threads_started":1}
I20260809 01:04:16.708604 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:16.686420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:46607 (local address 127.0.0.1:49494)
0809 01:04:16.700211 (+ 13791us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:16.700250 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:16.700409 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:16.701073 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:16.701080 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:16.701111 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:16.701432 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:16.701441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.703133 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.703138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:16.705841 (+ 2703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:16.705856 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.707741 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.707750 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:16.707918 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:16.707948 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 01:04:16.708023 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 01:04:16.708173 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13633,"mutex_wait_us":3,"thread_start_us":13466,"threads_started":1}
I20260809 01:04:16.708881 25443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:16.686351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:41865 (local address 127.0.0.1:51572)
0809 01:04:16.699299 (+ 12948us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:16.699355 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:16.699560 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:16.701578 (+ 2018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:16.701583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:16.701626 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:16.702211 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:16.702226 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.703857 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.703863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:16.705777 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:16.705793 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.707357 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.707363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:16.707573 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:16.707633 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 01:04:16.707782 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 01:04:16.707959 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12596,"mutex_wait_us":90,"spinlock_wait_cycles":3456,"thread_start_us":12436,"threads_started":1}
I20260809 01:04:16.709342 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:16.686877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:45373 (local address 127.0.0.1:54564)
0809 01:04:16.700530 (+ 13653us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:16.700551 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:16.700674 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:16.701578 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:16.701583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:16.701617 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:16.702236 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:16.702248 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.704259 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.704270 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:16.706507 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:16.706522 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.708569 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.708577 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:16.708784 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:16.708816 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 01:04:16.708899 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 01:04:16.709023 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13512,"mutex_wait_us":47,"spinlock_wait_cycles":1280,"thread_start_us":13331,"threads_started":1}
I20260809 01:04:16.709391 25448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:16.687340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49494 (local address 127.4.82.131:46607)
0809 01:04:16.698814 (+ 11474us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:16.698823 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:16.700547 (+ 1724us) server_negotiation.cc:407] Connection header received
0809 01:04:16.700658 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:16.700665 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:16.700872 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:16.701058 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:16.701605 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.702973 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.706029 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.707228 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.707373 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:16.707595 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:16.708827 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0809 01:04:16.709006 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10713,"thread_start_us":10519,"threads_started":1}
I20260809 01:04:16.710198 25447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:16.687359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54564 (local address 127.4.82.129:45373)
0809 01:04:16.697628 (+ 10269us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:16.697639 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:16.700829 (+ 3190us) server_negotiation.cc:407] Connection header received
0809 01:04:16.700927 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:16.700933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:16.701146 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:16.701347 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:16.702453 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.704083 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.706704 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:16.708032 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:16.708184 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:16.708396 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:16.709629 (+ 1233us) server_negotiation.cc:299] Negotiation successful
0809 01:04:16.709824 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10030,"thread_start_us":9860,"threads_started":1}
I20260809 01:04:16.712759 25362 tablet_service.cc:1467] Processing CreateTablet for tablet 3c094390d0084d5cb5a9a27471ba5edf (DEFAULT_TABLE table=test-table [id=682a04bd0fdf489f8634bdaba9e4ca15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:16.712813 25226 tablet_service.cc:1467] Processing CreateTablet for tablet 3c094390d0084d5cb5a9a27471ba5edf (DEFAULT_TABLE table=test-table [id=682a04bd0fdf489f8634bdaba9e4ca15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:16.713428 25088 tablet_service.cc:1467] Processing CreateTablet for tablet 3c094390d0084d5cb5a9a27471ba5edf (DEFAULT_TABLE table=test-table [id=682a04bd0fdf489f8634bdaba9e4ca15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:16.714092 25362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c094390d0084d5cb5a9a27471ba5edf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:16.714195 25226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c094390d0084d5cb5a9a27471ba5edf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:16.714720 25088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c094390d0084d5cb5a9a27471ba5edf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:16.725760 25449 tablet_bootstrap.cc:492] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c: Bootstrap starting.
I20260809 01:04:16.729256 25449 tablet_bootstrap.cc:654] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:16.730274 25449 log.cc:824] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:16.732177 25450 tablet_bootstrap.cc:492] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205: Bootstrap starting.
I20260809 01:04:16.733083 25451 tablet_bootstrap.cc:492] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff: Bootstrap starting.
I20260809 01:04:16.736025 25450 tablet_bootstrap.cc:654] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:16.736889 25451 tablet_bootstrap.cc:654] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:16.737031 25450 log.cc:824] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:16.738051 25449 tablet_bootstrap.cc:492] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c: No bootstrap required, opened a new log
I20260809 01:04:16.738317 25449 ts_tablet_manager.cc:1397] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260809 01:04:16.738368 25451 log.cc:824] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:16.741716 25450 tablet_bootstrap.cc:492] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205: No bootstrap required, opened a new log
I20260809 01:04:16.741717 25451 tablet_bootstrap.cc:492] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff: No bootstrap required, opened a new log
I20260809 01:04:16.741986 25451 ts_tablet_manager.cc:1397] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260809 01:04:16.741986 25450 ts_tablet_manager.cc:1397] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260809 01:04:16.745847 25449 raft_consensus.cc:348] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d486a08f2984a0b8b755f5d67547205" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41865 } } peers { permanent_uuid: "1ef514f7194d4355bc6706e207863b5c" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 46607 } } peers { permanent_uuid: "6f03be45d9814ddd91ec96587175deff" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45373 } }
I20260809 01:04:16.746218 25449 raft_consensus.cc:374] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:16.746340 25449 raft_consensus.cc:729] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef514f7194d4355bc6706e207863b5c, State: Initialized, Role: FOLLOWER
I20260809 01:04:16.746932 25449 consensus_queue.cc:260] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d486a08f2984a0b8b755f5d67547205" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41865 } } peers { permanent_uuid: "1ef514f7194d4355bc6706e207863b5c" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 46607 } } peers { permanent_uuid: "6f03be45d9814ddd91ec96587175deff" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45373 } }
I20260809 01:04:16.748101 25430 heartbeater.cc:503] Master 127.4.82.190:34911 was elected leader, sending a full tablet report...
I20260809 01:04:16.748706 25449 ts_tablet_manager.cc:1428] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c: Time spent starting tablet: real 0.010s user 0.008s sys 0.004s
I20260809 01:04:16.752800 25450 raft_consensus.cc:348] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d486a08f2984a0b8b755f5d67547205" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41865 } } peers { permanent_uuid: "1ef514f7194d4355bc6706e207863b5c" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 46607 } } peers { permanent_uuid: "6f03be45d9814ddd91ec96587175deff" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45373 } }
I20260809 01:04:16.752856 25451 raft_consensus.cc:348] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d486a08f2984a0b8b755f5d67547205" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41865 } } peers { permanent_uuid: "1ef514f7194d4355bc6706e207863b5c" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 46607 } } peers { permanent_uuid: "6f03be45d9814ddd91ec96587175deff" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45373 } }
I20260809 01:04:16.753261 25450 raft_consensus.cc:374] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:16.753261 25451 raft_consensus.cc:374] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:16.753362 25450 raft_consensus.cc:729] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d486a08f2984a0b8b755f5d67547205, State: Initialized, Role: FOLLOWER
I20260809 01:04:16.753362 25451 raft_consensus.cc:729] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f03be45d9814ddd91ec96587175deff, State: Initialized, Role: FOLLOWER
I20260809 01:04:16.753916 25450 consensus_queue.cc:260] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d486a08f2984a0b8b755f5d67547205" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41865 } } peers { permanent_uuid: "1ef514f7194d4355bc6706e207863b5c" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 46607 } } peers { permanent_uuid: "6f03be45d9814ddd91ec96587175deff" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45373 } }
I20260809 01:04:16.753989 25451 consensus_queue.cc:260] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d486a08f2984a0b8b755f5d67547205" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41865 } } peers { permanent_uuid: "1ef514f7194d4355bc6706e207863b5c" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 46607 } } peers { permanent_uuid: "6f03be45d9814ddd91ec96587175deff" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45373 } }
I20260809 01:04:16.755519 25292 heartbeater.cc:503] Master 127.4.82.190:34911 was elected leader, sending a full tablet report...
I20260809 01:04:16.755820 25451 ts_tablet_manager.cc:1428] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260809 01:04:16.756070 25450 ts_tablet_manager.cc:1428] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
W20260809 01:04:16.838785 25155 tablet.cc:2369] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 01:04:16.846067 25293 tablet.cc:2369] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 01:04:16.870743 25431 tablet.cc:2369] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:04:17.035045 25457 raft_consensus.cc:482] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:04:17.035413 25457 raft_consensus.cc:504] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d486a08f2984a0b8b755f5d67547205" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41865 } } peers { permanent_uuid: "1ef514f7194d4355bc6706e207863b5c" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 46607 } } peers { permanent_uuid: "6f03be45d9814ddd91ec96587175deff" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45373 } }
I20260809 01:04:17.037145 25457 leader_election.cc:288] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d486a08f2984a0b8b755f5d67547205 (127.4.82.130:41865), 1ef514f7194d4355bc6706e207863b5c (127.4.82.131:46607)
I20260809 01:04:17.044150 25448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:17.037734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:56937 (local address 127.4.82.131:46607)
0809 01:04:17.038243 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:17.038252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:17.038568 (+ 316us) server_negotiation.cc:407] Connection header received
0809 01:04:17.038858 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:17.038864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:17.038943 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:17.039043 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:17.039707 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.040458 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.042332 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.043081 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.043270 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:17.043383 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:17.043737 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 01:04:17.043877 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320}
I20260809 01:04:17.044334 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:17.037682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:59381 (local address 127.4.82.130:41865)
0809 01:04:17.038028 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:17.038034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:17.038754 (+ 720us) server_negotiation.cc:407] Connection header received
0809 01:04:17.038968 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:17.038971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:17.039031 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:17.039128 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:17.039885 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.040570 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.042470 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.043232 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.043453 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:17.043542 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:17.043998 (+ 456us) server_negotiation.cc:299] Negotiation successful
0809 01:04:17.044097 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260809 01:04:17.044477 25459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:17.037649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:41865 (local address 127.4.82.129:59381)
0809 01:04:17.038637 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:17.038651 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:17.038820 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:17.039143 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:17.039147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:17.039236 (+ 89us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:17.039725 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:17.039736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.040716 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.040721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:17.042310 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:17.042322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.043652 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.043661 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:17.043831 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:17.043863 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 01:04:17.043950 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 01:04:17.044081 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"spinlock_wait_cycles":23296,"thread_start_us":684,"threads_started":1}
I20260809 01:04:17.044870 25458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:17.037445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:46607 (local address 127.4.82.129:56937)
0809 01:04:17.038406 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:17.038439 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:17.038695 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:17.039064 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:17.039069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:17.039094 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:17.039556 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:17.039569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.040590 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.040597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:17.042163 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:17.042175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.043322 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.043329 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:17.043521 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:17.043561 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 01:04:17.044362 (+ 801us) client_negotiation.cc:240] Negotiation successful
0809 01:04:17.044524 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":501,"threads_started":1}
I20260809 01:04:17.045524 25246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c094390d0084d5cb5a9a27471ba5edf" candidate_uuid: "6f03be45d9814ddd91ec96587175deff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d486a08f2984a0b8b755f5d67547205" is_pre_election: true
I20260809 01:04:17.045789 25383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c094390d0084d5cb5a9a27471ba5edf" candidate_uuid: "6f03be45d9814ddd91ec96587175deff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef514f7194d4355bc6706e207863b5c" is_pre_election: true
I20260809 01:04:17.046002 25246 raft_consensus.cc:2454] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f03be45d9814ddd91ec96587175deff in term 0.
I20260809 01:04:17.046260 25383 raft_consensus.cc:2454] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f03be45d9814ddd91ec96587175deff in term 0.
I20260809 01:04:17.046700 25039 leader_election.cc:302] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f03be45d9814ddd91ec96587175deff, 9d486a08f2984a0b8b755f5d67547205; no voters:
I20260809 01:04:17.047211 25457 raft_consensus.cc:2781] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:04:17.047436 25457 raft_consensus.cc:482] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:04:17.047559 25457 raft_consensus.cc:3037] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:17.049932 25457 raft_consensus.cc:504] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d486a08f2984a0b8b755f5d67547205" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41865 } } peers { permanent_uuid: "1ef514f7194d4355bc6706e207863b5c" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 46607 } } peers { permanent_uuid: "6f03be45d9814ddd91ec96587175deff" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45373 } }
I20260809 01:04:17.050655 25457 leader_election.cc:288] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [CANDIDATE]: Term 1 election: Requested vote from peers 9d486a08f2984a0b8b755f5d67547205 (127.4.82.130:41865), 1ef514f7194d4355bc6706e207863b5c (127.4.82.131:46607)
I20260809 01:04:17.051124 25246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c094390d0084d5cb5a9a27471ba5edf" candidate_uuid: "6f03be45d9814ddd91ec96587175deff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d486a08f2984a0b8b755f5d67547205"
I20260809 01:04:17.051224 25383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c094390d0084d5cb5a9a27471ba5edf" candidate_uuid: "6f03be45d9814ddd91ec96587175deff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ef514f7194d4355bc6706e207863b5c"
I20260809 01:04:17.051378 25246 raft_consensus.cc:3037] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:17.051491 25383 raft_consensus.cc:3037] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:17.053787 25246 raft_consensus.cc:2454] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f03be45d9814ddd91ec96587175deff in term 1.
I20260809 01:04:17.053844 25383 raft_consensus.cc:2454] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f03be45d9814ddd91ec96587175deff in term 1.
I20260809 01:04:17.054268 25039 leader_election.cc:302] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f03be45d9814ddd91ec96587175deff, 9d486a08f2984a0b8b755f5d67547205; no voters:
I20260809 01:04:17.054597 25457 raft_consensus.cc:2781] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:17.055063 25457 raft_consensus.cc:686] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [term 1 LEADER]: Becoming Leader. State: Replica: 6f03be45d9814ddd91ec96587175deff, State: Running, Role: LEADER
I20260809 01:04:17.055600 25457 consensus_queue.cc:237] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d486a08f2984a0b8b755f5d67547205" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41865 } } peers { permanent_uuid: "1ef514f7194d4355bc6706e207863b5c" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 46607 } } peers { permanent_uuid: "6f03be45d9814ddd91ec96587175deff" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45373 } }
I20260809 01:04:17.060061 24956 catalog_manager.cc:5495] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f03be45d9814ddd91ec96587175deff (127.4.82.129). New cstate: current_term: 1 leader_uuid: "6f03be45d9814ddd91ec96587175deff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d486a08f2984a0b8b755f5d67547205" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ef514f7194d4355bc6706e207863b5c" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f03be45d9814ddd91ec96587175deff" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 45373 } health_report { overall_health: HEALTHY } } }
I20260809 01:04:17.151970 25447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:17.146722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54580 (local address 127.4.82.129:45373)
0809 01:04:17.146973 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:17.146978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:17.146997 (+ 19us) server_negotiation.cc:407] Connection header received
0809 01:04:17.147181 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:17.147185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:17.147278 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:17.147451 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:04:17.147827 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.148524 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.149876 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.150284 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.150371 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:17.150785 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:04:17.151407 (+ 622us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:04:17.151512 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:17.151614 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 01:04:17.151679 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 01:04:17.160353 25246 raft_consensus.cc:1261] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205 [term 1 FOLLOWER]: Refusing update from remote peer 6f03be45d9814ddd91ec96587175deff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:04:17.160353 25383 raft_consensus.cc:1261] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c [term 1 FOLLOWER]: Refusing update from remote peer 6f03be45d9814ddd91ec96587175deff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:04:17.161413 25460 consensus_queue.cc:1035] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ef514f7194d4355bc6706e207863b5c" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:04:17.161799 25457 consensus_queue.cc:1035] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d486a08f2984a0b8b755f5d67547205" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:04:17.172180 25463 mvcc.cc:205] Tried to move back new op lower bound from 7316428624517402624 to 7316428624103555072. Current Snapshot: MvccSnapshot[applied={T|T < 7316428624517402624}]
I20260809 01:04:17.174448 25465 mvcc.cc:205] Tried to move back new op lower bound from 7316428624517402624 to 7316428624103555072. Current Snapshot: MvccSnapshot[applied={T|T < 7316428624517402624}]
I20260809 01:04:17.186661 24957 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:04:17.186978 24957 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:04:17.190521 24957 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1987cb9a7b8d41159d8a653bbb737398: Sending DeleteTablet for 3 replicas of tablet 3c094390d0084d5cb5a9a27471ba5edf
I20260809 01:04:17.191679 25362 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c094390d0084d5cb5a9a27471ba5edf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:17 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:49494
I20260809 01:04:17.191668 25088 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c094390d0084d5cb5a9a27471ba5edf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:17 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:54564
I20260809 01:04:17.191653 25226 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c094390d0084d5cb5a9a27471ba5edf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:17 UTC) from {username='kudu', principal='kudu/127.4.82.190@KRBTEST.COM'} at 127.0.0.1:51572
I20260809 01:04:17.194380 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 25021
I20260809 01:04:17.220721 25473 tablet_replica.cc:330] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff: stopping tablet replica
I20260809 01:04:17.221360 25473 raft_consensus.cc:2229] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:04:17.222198 25473 raft_consensus.cc:2258] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:17.223073 25474 tablet_replica.cc:330] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c: stopping tablet replica
I20260809 01:04:17.223515 25474 raft_consensus.cc:2229] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:04:17.223979 25474 raft_consensus.cc:2258] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:17.224781 25475 tablet_replica.cc:330] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205: stopping tablet replica
I20260809 01:04:17.225198 25475 raft_consensus.cc:2229] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:04:17.225661 25475 raft_consensus.cc:2258] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:17.227022 25473 ts_tablet_manager.cc:1905] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:17.228044 25475 ts_tablet_manager.cc:1905] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 01:04:17 dist-test-slave-07c2 krb5kdc[24912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.129@KRBTEST.COM
I20260809 01:04:17.232648 25474 ts_tablet_manager.cc:1905] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:17.234980 25473 ts_tablet_manager.cc:1918] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:17.234966 25475 ts_tablet_manager.cc:1918] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:17.235131 25475 log.cc:1196] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/3c094390d0084d5cb5a9a27471ba5edf
I20260809 01:04:17.235316 25473 log.cc:1196] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-0/wal/wals/3c094390d0084d5cb5a9a27471ba5edf
I20260809 01:04:17.235766 25475 ts_tablet_manager.cc:1939] T 3c094390d0084d5cb5a9a27471ba5edf P 9d486a08f2984a0b8b755f5d67547205: Deleting consensus metadata
I20260809 01:04:17.236016 25473 ts_tablet_manager.cc:1939] T 3c094390d0084d5cb5a9a27471ba5edf P 6f03be45d9814ddd91ec96587175deff: Deleting consensus metadata
I20260809 01:04:17.237848 24941 catalog_manager.cc:4841] TS 9d486a08f2984a0b8b755f5d67547205 (127.4.82.130:41865): tablet 3c094390d0084d5cb5a9a27471ba5edf (table test-table [id=682a04bd0fdf489f8634bdaba9e4ca15]) successfully deleted
I20260809 01:04:17.237848 24944 catalog_manager.cc:4841] TS 6f03be45d9814ddd91ec96587175deff (127.4.82.129:45373): tablet 3c094390d0084d5cb5a9a27471ba5edf (table test-table [id=682a04bd0fdf489f8634bdaba9e4ca15]) successfully deleted
I20260809 01:04:17.239462 25474 ts_tablet_manager.cc:1918] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:17.239624 25474 log.cc:1196] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/3c094390d0084d5cb5a9a27471ba5edf
I20260809 01:04:17.240145 25474 ts_tablet_manager.cc:1939] T 3c094390d0084d5cb5a9a27471ba5edf P 1ef514f7194d4355bc6706e207863b5c: Deleting consensus metadata
I20260809 01:04:17.241866 24942 catalog_manager.cc:4841] TS 1ef514f7194d4355bc6706e207863b5c (127.4.82.131:46607): tablet 3c094390d0084d5cb5a9a27471ba5edf (table test-table [id=682a04bd0fdf489f8634bdaba9e4ca15]) successfully deleted
I20260809 01:04:17.245997 25447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:17.195591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54584 (local address 127.4.82.129:45373)
0809 01:04:17.195750 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:17.195758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:17.212952 (+ 17194us) server_negotiation.cc:407] Connection header received
0809 01:04:17.213571 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:17.213581 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:17.213707 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:17.213805 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:17.216525 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.218115 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.225896 (+ 7781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.226644 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.226766 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:17.239437 (+ 12671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:17.239448 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:17.239461 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:17.239500 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:17.242941 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:17.243784 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:17.243791 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:17.243793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:17.243939 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:17.244378 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:17.244382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:17.244384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:17.244728 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:17.245055 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:17.245377 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 01:04:17.245627 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 01:04:17.246819 25149 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:17.345299 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 25021
I20260809 01:04:17.367195 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 25160
Aug 09 01:04:17 dist-test-slave-07c2 krb5kdc[24912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.130@KRBTEST.COM
I20260809 01:04:17.381914 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:17.367981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51576 (local address 127.4.82.130:41865)
0809 01:04:17.368149 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:17.368154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:17.368236 (+ 82us) server_negotiation.cc:407] Connection header received
0809 01:04:17.368379 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:17.368382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:17.368444 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:17.368527 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:17.369832 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.370529 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.371976 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.372349 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.372441 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:17.376577 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:17.376590 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:17.376604 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:17.376644 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:17.379531 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:17.380160 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:17.380168 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:17.380170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:17.380299 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:17.380679 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:17.380682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:17.380684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:17.381029 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:17.381237 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:17.381482 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 01:04:17.381633 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 01:04:17.382552 25287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:17.472932 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 25160
I20260809 01:04:17.492221 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 25297
Aug 09 01:04:17 dist-test-slave-07c2 krb5kdc[24912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.131@KRBTEST.COM
I20260809 01:04:17.508167 25448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:17.492990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49502 (local address 127.4.82.131:46607)
0809 01:04:17.493155 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:17.493160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:17.493180 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:04:17.493339 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:17.493343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:17.493412 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:17.493506 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:17.495053 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.495917 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.497186 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:17.497544 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:17.497636 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:17.502430 (+ 4794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:17.502445 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:17.502463 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:17.502496 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:17.505362 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:17.506050 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:17.506058 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:17.506060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:17.506212 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:17.506606 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:17.506612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:17.506615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:17.506960 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:17.507266 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:17.507600 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 01:04:17.507828 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 01:04:17.508831 25425 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:17.600714 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 25297
I20260809 01:04:17.619130 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 24925
I20260809 01:04:17.620281 24989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:17.732229 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 24925
2026-08-09T01:04:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4663 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T01:04:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:17Z Disabled control of system clock
I20260809 01:04:17.823074 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34721
--webserver_interface=127.4.82.190
--webserver_port=0
--builtin_ntp_servers=127.4.82.148:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:34721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 01:04:18.155020 25486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:18.155393 25486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:18.155486 25486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:18.162613 25486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:18.162711 25486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:18.162751 25486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:18.162817 25486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:18.162863 25486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:18.172551 25486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:34721
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:34721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:18.173769 25486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:18.175531 25486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:18.187273 25492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:18.187206 25495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:18.187247 25493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:18.188490 25486 server_base.cc:1034] running on GCE node
I20260809 01:04:18.189188 25486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:18.190486 25486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:18.191695 25486 hybrid_clock.cc:648] HybridClock initialized: now 1786237458191645 us; error 31 us; skew 500 ppm
I20260809 01:04:18.192009 25486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:04:18.194125 25486 webserver.cc:460] Webserver started at http://127.4.82.190:34239/ using document root <none> and password file <none>
I20260809 01:04:18.194732 25486 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:18.194818 25486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:18.195154 25486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:18.198102 25486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "542244e8190e427db7c99d398c73dc6f"
format_stamp: "Formatted at 2026-08-09 01:04:18 on dist-test-slave-07c2"
server_key: "72d0697b62da59549ff89845f3c13695"
server_key_iv: "c69b5b544816b88190db0d0196cf7345"
server_key_version: "encryptionkey@0"
I20260809 01:04:18.198855 25486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "542244e8190e427db7c99d398c73dc6f"
format_stamp: "Formatted at 2026-08-09 01:04:18 on dist-test-slave-07c2"
server_key: "72d0697b62da59549ff89845f3c13695"
server_key_iv: "c69b5b544816b88190db0d0196cf7345"
server_key_version: "encryptionkey@0"
I20260809 01:04:18.203779 25486 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 01:04:18.207046 25501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:18.208381 25486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:04:18.208578 25486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "542244e8190e427db7c99d398c73dc6f"
format_stamp: "Formatted at 2026-08-09 01:04:18 on dist-test-slave-07c2"
server_key: "72d0697b62da59549ff89845f3c13695"
server_key_iv: "c69b5b544816b88190db0d0196cf7345"
server_key_version: "encryptionkey@0"
I20260809 01:04:18.208788 25486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:18.225262 25486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:18.226095 25486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:18.226390 25486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:18.245116 25486 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:34721
I20260809 01:04:18.245151 25556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:34721 every 8 connection(s)
I20260809 01:04:18.246454 25486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:18.248245 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 25486
I20260809 01:04:18.248425 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:18.248890 4426 external_mini_cluster.cc:1428] Setting key 58fa435148f0737eb5d2b26fd9eb1cbf
I20260809 01:04:18.250653 25557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:18.262270 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:18.250812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47596 (local address 127.4.82.190:34721)
0809 01:04:18.251819 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:18.251829 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:18.251867 (+ 38us) server_negotiation.cc:407] Connection header received
0809 01:04:18.253436 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:18.253454 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:18.253883 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:18.254236 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:18.254832 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:18.256172 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:18.258068 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:18.258693 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:18.258819 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:18.260053 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:18.260060 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:18.260072 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:18.260608 (+ 536us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:18.260690 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:18.260721 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:18.260814 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:18.261042 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 01:04:18.261368 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":386,"threads_started":1}
I20260809 01:04:18.263636 25557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f: Bootstrap starting.
I20260809 01:04:18.266629 25557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:18.267679 25557 log.cc:824] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:18.270316 25557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f: No bootstrap required, opened a new log
I20260809 01:04:18.277486 25557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542244e8190e427db7c99d398c73dc6f" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34721 } }
I20260809 01:04:18.277850 25557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:18.277961 25557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542244e8190e427db7c99d398c73dc6f, State: Initialized, Role: FOLLOWER
I20260809 01:04:18.278554 25557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542244e8190e427db7c99d398c73dc6f" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34721 } }
I20260809 01:04:18.278802 25557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:18.278960 25557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:18.279160 25557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:18.281279 25557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542244e8190e427db7c99d398c73dc6f" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34721 } }
I20260809 01:04:18.281800 25557 leader_election.cc:302] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 542244e8190e427db7c99d398c73dc6f; no voters:
I20260809 01:04:18.282533 25557 leader_election.cc:288] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:18.282658 25562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:18.283432 25562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [term 1 LEADER]: Becoming Leader. State: Replica: 542244e8190e427db7c99d398c73dc6f, State: Running, Role: LEADER
I20260809 01:04:18.283968 25562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542244e8190e427db7c99d398c73dc6f" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34721 } }
I20260809 01:04:18.284765 25557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:18.286823 25564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "542244e8190e427db7c99d398c73dc6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542244e8190e427db7c99d398c73dc6f" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34721 } } }
I20260809 01:04:18.286923 25565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 542244e8190e427db7c99d398c73dc6f. Latest consensus state: current_term: 1 leader_uuid: "542244e8190e427db7c99d398c73dc6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542244e8190e427db7c99d398c73dc6f" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 34721 } } }
I20260809 01:04:18.287348 25565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:18.287377 25564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:18.296708 25568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:18.302604 25568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:18.311854 25568 catalog_manager.cc:1347] Generated new cluster ID: f0ae9a6c1c8843a29bb0cffa2c879edf
I20260809 01:04:18.311996 25568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:18.334545 25568 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:18.335891 25568 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:18.342557 25568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f: Generated new TSK 0
I20260809 01:04:18.343372 25568 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:18.359558 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.129:0
--local_ip_for_outbound_sockets=127.4.82.129
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34721
--builtin_ntp_servers=127.4.82.148:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 01:04:18.715152 25582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:18.715489 25582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:18.715694 25582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:18.722929 25582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:18.723037 25582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:18.723210 25582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.129
I20260809 01:04:18.732568 25582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.82.129
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.129
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:18.733949 25582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:18.735901 25582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:18.749238 25587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:18.749349 25588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:18.749392 25590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:18.750267 25582 server_base.cc:1034] running on GCE node
I20260809 01:04:18.750871 25582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:18.751935 25582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:18.753173 25582 hybrid_clock.cc:648] HybridClock initialized: now 1786237458753122 us; error 45 us; skew 500 ppm
I20260809 01:04:18.753495 25582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:04:18.755816 25582 webserver.cc:460] Webserver started at http://127.4.82.129:42441/ using document root <none> and password file <none>
I20260809 01:04:18.756525 25582 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:18.756664 25582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:18.756989 25582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:18.760179 25582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/data/instance:
uuid: "864fb157a1f64feab8c6ce01fa846b5c"
format_stamp: "Formatted at 2026-08-09 01:04:18 on dist-test-slave-07c2"
server_key: "a14144bc2fc94964497313d9021a0aec"
server_key_iv: "5f9f2acd3dc3c2e13b8b4c7dba9007cd"
server_key_version: "encryptionkey@0"
I20260809 01:04:18.761001 25582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance:
uuid: "864fb157a1f64feab8c6ce01fa846b5c"
format_stamp: "Formatted at 2026-08-09 01:04:18 on dist-test-slave-07c2"
server_key: "a14144bc2fc94964497313d9021a0aec"
server_key_iv: "5f9f2acd3dc3c2e13b8b4c7dba9007cd"
server_key_version: "encryptionkey@0"
I20260809 01:04:18.766501 25582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 01:04:18.770078 25596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:18.771473 25582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:04:18.771652 25582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/wal
uuid: "864fb157a1f64feab8c6ce01fa846b5c"
format_stamp: "Formatted at 2026-08-09 01:04:18 on dist-test-slave-07c2"
server_key: "a14144bc2fc94964497313d9021a0aec"
server_key_iv: "5f9f2acd3dc3c2e13b8b4c7dba9007cd"
server_key_version: "encryptionkey@0"
I20260809 01:04:18.771879 25582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:18.799674 25582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:18.800683 25582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:18.801024 25582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:18.802070 25582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:18.803716 25582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:18.803853 25582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:18.804023 25582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:18.804129 25582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:18.839716 25582 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.129:36407
I20260809 01:04:18.839805 25713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.129:36407 every 8 connection(s)
I20260809 01:04:18.841063 25582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/data/info.pb
I20260809 01:04:18.844292 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 25582
I20260809 01:04:18.844498 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-0/wal/instance
I20260809 01:04:18.844980 4426 external_mini_cluster.cc:1428] Setting key 8b6b6e9605e3634e635939f3283020c6
I20260809 01:04:18.848297 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.130:0
--local_ip_for_outbound_sockets=127.4.82.130
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34721
--builtin_ntp_servers=127.4.82.148:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 01:04:18.856305 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:18.843794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.129:42741 (local address 127.4.82.190:34721)
0809 01:04:18.847080 (+ 3286us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:18.847086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:18.847106 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:04:18.848625 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:18.848630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:18.848697 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:18.848793 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:18.850173 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:18.850973 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:18.853435 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:18.853842 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:18.853937 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:18.855568 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:18.855573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:18.855576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:18.855594 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:18.855636 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:18.855648 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:18.855714 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:18.855979 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 01:04:18.856052 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3159,"thread_start_us":3056,"threads_started":1}
I20260809 01:04:18.857688 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:18.844652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34721 (local address 127.4.82.129:42741)
0809 01:04:18.845864 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:18.845912 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:18.848250 (+ 2338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:18.849047 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:18.849061 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:18.849176 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:18.849988 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:18.850014 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:18.851144 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:18.851151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:18.853268 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:18.853287 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:18.854053 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:18.854062 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:18.854412 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:18.855187 (+ 775us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:04:18.855192 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:18.855256 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:04:18.855376 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:18.855741 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:18.855756 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:18.855766 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 01:04:18.855952 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 01:04:18.856346 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":542,"threads_started":1}
I20260809 01:04:18.859339 25714 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34721
I20260809 01:04:18.859611 25714 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:18.860311 25714 heartbeater.cc:511] Master 127.4.82.190:34721 requested a full tablet report, sending...
I20260809 01:04:18.862206 25519 ts_manager.cc:194] Registered new tserver with Master: 864fb157a1f64feab8c6ce01fa846b5c (127.4.82.129:36407)
I20260809 01:04:18.863703 25519 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.82.129:42741
W20260809 01:04:19.178471 25718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:19.178730 25718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:19.178902 25718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:19.186473 25718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:19.186566 25718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:19.186798 25718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.130
I20260809 01:04:19.195784 25718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.82.130
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.130
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:19.197000 25718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:19.198731 25718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:19.211633 25724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:19.211699 25726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:19.211733 25723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:19.212703 25718 server_base.cc:1034] running on GCE node
I20260809 01:04:19.213366 25718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:19.214330 25718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:19.215555 25718 hybrid_clock.cc:648] HybridClock initialized: now 1786237459215513 us; error 66 us; skew 500 ppm
I20260809 01:04:19.215880 25718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:04:19.218125 25718 webserver.cc:460] Webserver started at http://127.4.82.130:41929/ using document root <none> and password file <none>
I20260809 01:04:19.218788 25718 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:19.218920 25718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:19.219240 25718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:19.222081 25718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/data/instance:
uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0"
format_stamp: "Formatted at 2026-08-09 01:04:19 on dist-test-slave-07c2"
server_key: "5977129aa766578ca8369ba91513c4d1"
server_key_iv: "334e4aaa70cdfaeba99d4b85708fc990"
server_key_version: "encryptionkey@0"
I20260809 01:04:19.222859 25718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance:
uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0"
format_stamp: "Formatted at 2026-08-09 01:04:19 on dist-test-slave-07c2"
server_key: "5977129aa766578ca8369ba91513c4d1"
server_key_iv: "334e4aaa70cdfaeba99d4b85708fc990"
server_key_version: "encryptionkey@0"
I20260809 01:04:19.227694 25718 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 01:04:19.231092 25732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:19.232379 25718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:04:19.232546 25718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/wal
uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0"
format_stamp: "Formatted at 2026-08-09 01:04:19 on dist-test-slave-07c2"
server_key: "5977129aa766578ca8369ba91513c4d1"
server_key_iv: "334e4aaa70cdfaeba99d4b85708fc990"
server_key_version: "encryptionkey@0"
I20260809 01:04:19.232774 25718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:19.251827 25718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:19.252686 25718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:19.253003 25718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:19.254060 25718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:19.255767 25718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:19.255872 25718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:19.255962 25718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:19.256028 25718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:19.290088 25718 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.130:41471
I20260809 01:04:19.290176 25850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.130:41471 every 8 connection(s)
I20260809 01:04:19.291474 25718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/data/info.pb
I20260809 01:04:19.299688 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 25718
I20260809 01:04:19.299907 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/wal/instance
I20260809 01:04:19.300397 4426 external_mini_cluster.cc:1428] Setting key 735d38b08d4c7da6821cb1833f39eefb
I20260809 01:04:19.303205 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.82.131:0
--local_ip_for_outbound_sockets=127.4.82.131
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34721
--builtin_ntp_servers=127.4.82.148:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 01:04:19.304478 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.293966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.130:59763 (local address 127.4.82.190:34721)
0809 01:04:19.294148 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:19.294152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:19.295636 (+ 1484us) server_negotiation.cc:407] Connection header received
0809 01:04:19.297420 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:19.297426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:19.297518 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:19.297633 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:19.298674 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.299586 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.301407 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.301845 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.301942 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.303509 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:19.303514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:19.303517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:19.303535 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:19.303566 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:19.303578 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:19.303628 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:19.303917 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 01:04:19.304032 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 01:04:19.305301 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.294435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34721 (local address 127.4.82.130:59763)
0809 01:04:19.295480 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:19.295524 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:19.297147 (+ 1623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:19.297861 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:19.297869 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:19.297950 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:19.298493 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.298508 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.299743 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.299747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.301271 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.301283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.302080 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.302086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.302393 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.303117 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:04:19.303120 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:19.303156 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:04:19.303233 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:19.303705 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:19.303713 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:19.303720 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 01:04:19.303850 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 01:04:19.304161 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":292,"threads_started":1}
I20260809 01:04:19.306692 25851 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34721
I20260809 01:04:19.306998 25851 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:19.307906 25851 heartbeater.cc:511] Master 127.4.82.190:34721 requested a full tablet report, sending...
I20260809 01:04:19.309414 25519 ts_manager.cc:194] Registered new tserver with Master: 5a4875d2d8cc4d9eb075189ec0cda7d0 (127.4.82.130:41471)
I20260809 01:04:19.310186 25519 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.82.130:59763
W20260809 01:04:19.642750 25856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:19.643035 25856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:19.643195 25856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:19.650466 25856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:19.650574 25856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:19.650786 25856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.82.131
I20260809 01:04:19.660070 25856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.82.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.82.131
--webserver_port=0
--tserver_master_addrs=127.4.82.190:34721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.82.131
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:19.661393 25856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:19.663267 25856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:19.676724 25861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:19.676637 25864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:19.676743 25862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:19.682804 25856 server_base.cc:1034] running on GCE node
I20260809 01:04:19.683496 25856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:19.684471 25856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:19.685699 25856 hybrid_clock.cc:648] HybridClock initialized: now 1786237459685639 us; error 47 us; skew 500 ppm
I20260809 01:04:19.686017 25856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:04:19.688212 25856 webserver.cc:460] Webserver started at http://127.4.82.131:40729/ using document root <none> and password file <none>
I20260809 01:04:19.688884 25856 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:19.689016 25856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:19.689345 25856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:19.692160 25856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/data/instance:
uuid: "cffbf53971a84a3bbf411cb227f7961f"
format_stamp: "Formatted at 2026-08-09 01:04:19 on dist-test-slave-07c2"
server_key: "ceb0ac6bbf3d5dd28392e2942b409f85"
server_key_iv: "4201e0e6506963d6102601d99e3a3e5a"
server_key_version: "encryptionkey@0"
I20260809 01:04:19.692939 25856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance:
uuid: "cffbf53971a84a3bbf411cb227f7961f"
format_stamp: "Formatted at 2026-08-09 01:04:19 on dist-test-slave-07c2"
server_key: "ceb0ac6bbf3d5dd28392e2942b409f85"
server_key_iv: "4201e0e6506963d6102601d99e3a3e5a"
server_key_version: "encryptionkey@0"
I20260809 01:04:19.697752 25856 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 01:04:19.700968 25871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:19.702175 25856 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 01:04:19.702348 25856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/wal
uuid: "cffbf53971a84a3bbf411cb227f7961f"
format_stamp: "Formatted at 2026-08-09 01:04:19 on dist-test-slave-07c2"
server_key: "ceb0ac6bbf3d5dd28392e2942b409f85"
server_key_iv: "4201e0e6506963d6102601d99e3a3e5a"
server_key_version: "encryptionkey@0"
I20260809 01:04:19.702553 25856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:19.730906 25856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:19.731861 25856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:19.732154 25856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:19.733060 25856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 01:04:19.734437 25856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 01:04:19.734557 25856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:19.734685 25856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 01:04:19.734773 25856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:19.767465 25856 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.131:44163
I20260809 01:04:19.767552 25990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.131:44163 every 8 connection(s)
I20260809 01:04:19.768733 25856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/data/info.pb
I20260809 01:04:19.774782 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 25856
I20260809 01:04:19.774973 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/wal/instance
I20260809 01:04:19.775566 4426 external_mini_cluster.cc:1428] Setting key e49a8641951777f8a9b8c8be016ab5af
I20260809 01:04:19.781109 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.771221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:60165 (local address 127.4.82.190:34721)
0809 01:04:19.771440 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:19.771445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:19.772747 (+ 1302us) server_negotiation.cc:407] Connection header received
0809 01:04:19.774496 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:19.774502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:19.774590 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:19.774719 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:19.776000 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.776970 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.778463 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.778864 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.779000 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.780266 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:19.780272 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:19.780277 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:19.780303 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:19.780333 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:19.780345 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:19.780441 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:19.780673 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 01:04:19.780778 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 01:04:19.782032 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.771744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:34721 (local address 127.4.82.131:60165)
0809 01:04:19.772601 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:19.772635 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:19.774229 (+ 1594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:19.774945 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:19.774957 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:19.775066 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:19.775787 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.775847 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.777110 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.777115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.778324 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.778335 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.779080 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.779086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.779382 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.779968 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 01:04:19.779970 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:19.780009 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 01:04:19.780075 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:19.780440 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:19.780451 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:19.780461 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 01:04:19.780643 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 01:04:19.780988 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":266,"threads_started":1}
I20260809 01:04:19.783254 25991 heartbeater.cc:347] Connected to a master server at 127.4.82.190:34721
I20260809 01:04:19.783543 25991 heartbeater.cc:464] Registering TS with master...
I20260809 01:04:19.784179 25991 heartbeater.cc:511] Master 127.4.82.190:34721 requested a full tablet report, sending...
I20260809 01:04:19.785573 25519 ts_manager.cc:194] Registered new tserver with Master: cffbf53971a84a3bbf411cb227f7961f (127.4.82.131:44163)
I20260809 01:04:19.786342 25519 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.82.131:60165
I20260809 01:04:19.790262 4426 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 01:04:19.800185 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.794309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55428 (local address 127.4.82.190:34721)
0809 01:04:19.794511 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:19.794518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:19.794954 (+ 436us) server_negotiation.cc:407] Connection header received
0809 01:04:19.795145 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:19.795151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:19.795230 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:19.795353 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:19.795860 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.796538 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.797869 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.798283 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.798418 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.799481 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:19.799493 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:19.799497 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:19.799513 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:19.799542 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:19.799554 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:19.799645 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:19.799808 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 01:04:19.799898 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 01:04:19.803581 25519 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 01:04:19.811040 25519 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 01:04:19.835264 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.822569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36407 (local address 127.0.0.1:57950)
0809 01:04:19.823319 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:19.823366 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:19.823575 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:19.824740 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:19.824751 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:19.824800 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:19.825267 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.825284 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.827840 (+ 2556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.827849 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.831009 (+ 3160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.831030 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.833986 (+ 2956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.833995 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.834311 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.834389 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 01:04:19.834539 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 01:04:19.834790 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":301,"threads_started":1}
I20260809 01:04:19.836622 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.822820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57950 (local address 127.4.82.129:36407)
0809 01:04:19.823432 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:19.823440 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:19.823472 (+ 32us) server_negotiation.cc:407] Connection header received
0809 01:04:19.823870 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:19.823878 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:19.824127 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:19.824361 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:19.825508 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.827618 (+ 2110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.831327 (+ 3709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.833345 (+ 2018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.833512 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.833823 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:19.835936 (+ 2113us) server_negotiation.cc:299] Negotiation successful
0809 01:04:19.836154 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":271,"threads_started":1}
I20260809 01:04:19.842756 25646 tablet_service.cc:1467] Processing CreateTablet for tablet e5c417e0bd924783b2281b9c211d1954 (DEFAULT_TABLE table=test-table [id=244d02f5d49e475ba03bc9c17464d977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:19.844296 25646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5c417e0bd924783b2281b9c211d1954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:19.847016 26006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.824066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:41471 (local address 127.0.0.1:58856)
0809 01:04:19.837611 (+ 13545us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:19.837635 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:19.837783 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:19.839374 (+ 1591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:19.839378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:19.839458 (+ 80us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:19.839785 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.839798 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.841964 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.841969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.844339 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.844353 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.846196 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.846206 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.846408 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.846441 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 01:04:19.846513 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 01:04:19.846666 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13356,"mutex_wait_us":221,"thread_start_us":15040,"threads_started":1}
I20260809 01:04:19.847400 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.823643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.131:44163 (local address 127.0.0.1:53678)
0809 01:04:19.835575 (+ 11932us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:19.835598 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:19.835729 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:19.840328 (+ 4599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:19.840333 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:19.840363 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:19.840550 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.840561 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.842685 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.842690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.844645 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.844663 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.846591 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.846598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.846785 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.846817 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 01:04:19.846881 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 01:04:19.847011 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11784,"mutex_wait_us":39,"thread_start_us":13310,"threads_started":1}
I20260809 01:04:19.848079 26010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.824705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.4.82.130:41471)
0809 01:04:19.838384 (+ 13679us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:19.838396 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:19.838433 (+ 37us) server_negotiation.cc:407] Connection header received
0809 01:04:19.838549 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:19.838560 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:19.838843 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:19.839989 (+ 1146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:19.840276 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.841818 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.844545 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.845966 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.847254 (+ 1288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.847516 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:19.847584 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 01:04:19.847750 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13471,"thread_start_us":13343,"threads_started":1}
I20260809 01:04:19.849392 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.823932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53678 (local address 127.4.82.131:44163)
0809 01:04:19.839809 (+ 15877us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:19.839815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:19.839840 (+ 25us) server_negotiation.cc:407] Connection header received
0809 01:04:19.839907 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:19.839912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:19.840103 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:19.840727 (+ 624us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:19.841021 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.842504 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.845083 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.846399 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.847719 (+ 1320us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.847970 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:19.848035 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 01:04:19.849032 (+ 997us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15747,"thread_start_us":20014,"threads_started":1}
I20260809 01:04:19.851438 25782 tablet_service.cc:1467] Processing CreateTablet for tablet e5c417e0bd924783b2281b9c211d1954 (DEFAULT_TABLE table=test-table [id=244d02f5d49e475ba03bc9c17464d977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:19.852690 25782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5c417e0bd924783b2281b9c211d1954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:19.852658 25922 tablet_service.cc:1467] Processing CreateTablet for tablet e5c417e0bd924783b2281b9c211d1954 (DEFAULT_TABLE table=test-table [id=244d02f5d49e475ba03bc9c17464d977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 01:04:19.853880 25922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5c417e0bd924783b2281b9c211d1954. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:19.862599 26011 tablet_bootstrap.cc:492] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c: Bootstrap starting.
I20260809 01:04:19.865269 26012 tablet_bootstrap.cc:492] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f: Bootstrap starting.
I20260809 01:04:19.866235 25714 heartbeater.cc:503] Master 127.4.82.190:34721 was elected leader, sending a full tablet report...
I20260809 01:04:19.866432 26011 tablet_bootstrap.cc:654] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:19.868062 26011 log.cc:824] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:19.869257 26012 tablet_bootstrap.cc:654] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:19.870613 26012 log.cc:824] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:19.872218 26014 tablet_bootstrap.cc:492] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0: Bootstrap starting.
I20260809 01:04:19.875546 26012 tablet_bootstrap.cc:492] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f: No bootstrap required, opened a new log
I20260809 01:04:19.875708 26011 tablet_bootstrap.cc:492] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c: No bootstrap required, opened a new log
I20260809 01:04:19.875866 26012 ts_tablet_manager.cc:1397] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20260809 01:04:19.875897 26014 tablet_bootstrap.cc:654] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:19.875970 26011 ts_tablet_manager.cc:1397] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.007s
I20260809 01:04:19.877082 26014 log.cc:824] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:19.883133 26012 raft_consensus.cc:348] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864fb157a1f64feab8c6ce01fa846b5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36407 } } peers { permanent_uuid: "cffbf53971a84a3bbf411cb227f7961f" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44163 } } peers { permanent_uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41471 } }
I20260809 01:04:19.883612 26012 raft_consensus.cc:374] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:19.883730 26012 raft_consensus.cc:729] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cffbf53971a84a3bbf411cb227f7961f, State: Initialized, Role: FOLLOWER
I20260809 01:04:19.884367 26012 consensus_queue.cc:260] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864fb157a1f64feab8c6ce01fa846b5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36407 } } peers { permanent_uuid: "cffbf53971a84a3bbf411cb227f7961f" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44163 } } peers { permanent_uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41471 } }
I20260809 01:04:19.885540 25991 heartbeater.cc:503] Master 127.4.82.190:34721 was elected leader, sending a full tablet report...
I20260809 01:04:19.886144 26012 ts_tablet_manager.cc:1428] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 01:04:19.887595 26014 tablet_bootstrap.cc:492] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0: No bootstrap required, opened a new log
I20260809 01:04:19.887894 26014 ts_tablet_manager.cc:1397] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.008s
I20260809 01:04:19.887979 26011 raft_consensus.cc:348] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864fb157a1f64feab8c6ce01fa846b5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36407 } } peers { permanent_uuid: "cffbf53971a84a3bbf411cb227f7961f" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44163 } } peers { permanent_uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41471 } }
I20260809 01:04:19.888417 26011 raft_consensus.cc:374] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:19.888520 26011 raft_consensus.cc:729] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 864fb157a1f64feab8c6ce01fa846b5c, State: Initialized, Role: FOLLOWER
I20260809 01:04:19.889202 26018 raft_consensus.cc:482] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 01:04:19.889099 26011 consensus_queue.cc:260] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864fb157a1f64feab8c6ce01fa846b5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36407 } } peers { permanent_uuid: "cffbf53971a84a3bbf411cb227f7961f" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44163 } } peers { permanent_uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41471 } }
I20260809 01:04:19.889544 26018 raft_consensus.cc:504] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864fb157a1f64feab8c6ce01fa846b5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36407 } } peers { permanent_uuid: "cffbf53971a84a3bbf411cb227f7961f" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44163 } } peers { permanent_uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41471 } }
I20260809 01:04:19.894500 26018 leader_election.cc:288] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 864fb157a1f64feab8c6ce01fa846b5c (127.4.82.129:36407), 5a4875d2d8cc4d9eb075189ec0cda7d0 (127.4.82.130:41471)
I20260809 01:04:19.898799 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.891671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36407 (local address 127.4.82.131:51013)
0809 01:04:19.891856 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:19.891877 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:19.891989 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:19.892592 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:19.892599 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:19.892627 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:19.892832 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.892847 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.894534 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.894540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.896335 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.896349 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.898038 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.898045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.898245 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.898280 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 01:04:19.898370 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 01:04:19.898505 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 01:04:19.899456 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.891803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:51013 (local address 127.4.82.129:36407)
0809 01:04:19.891982 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:19.891989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:19.892011 (+ 22us) server_negotiation.cc:407] Connection header received
0809 01:04:19.892114 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:19.892122 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:19.892228 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:19.892339 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:19.893026 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.894360 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.896542 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.897738 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.897885 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.897949 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:19.898990 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0809 01:04:19.899136 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 01:04:19.900859 26011 ts_tablet_manager.cc:1428] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c: Time spent starting tablet: real 0.025s user 0.022s sys 0.000s
I20260809 01:04:19.900820 25666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c417e0bd924783b2281b9c211d1954" candidate_uuid: "cffbf53971a84a3bbf411cb227f7961f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "864fb157a1f64feab8c6ce01fa846b5c" is_pre_election: true
I20260809 01:04:19.901394 25666 raft_consensus.cc:2454] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cffbf53971a84a3bbf411cb227f7961f in term 0.
I20260809 01:04:19.902251 25875 leader_election.cc:302] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 864fb157a1f64feab8c6ce01fa846b5c, cffbf53971a84a3bbf411cb227f7961f; no voters:
I20260809 01:04:19.903087 26014 raft_consensus.cc:348] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864fb157a1f64feab8c6ce01fa846b5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36407 } } peers { permanent_uuid: "cffbf53971a84a3bbf411cb227f7961f" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44163 } } peers { permanent_uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41471 } }
I20260809 01:04:19.903584 26014 raft_consensus.cc:374] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:19.903714 26014 raft_consensus.cc:729] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a4875d2d8cc4d9eb075189ec0cda7d0, State: Initialized, Role: FOLLOWER
I20260809 01:04:19.904300 26018 raft_consensus.cc:2781] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 01:04:19.904460 26018 raft_consensus.cc:482] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 01:04:19.904387 26014 consensus_queue.cc:260] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864fb157a1f64feab8c6ce01fa846b5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36407 } } peers { permanent_uuid: "cffbf53971a84a3bbf411cb227f7961f" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44163 } } peers { permanent_uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41471 } }
I20260809 01:04:19.904597 26018 raft_consensus.cc:3037] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:19.905539 25851 heartbeater.cc:503] Master 127.4.82.190:34721 was elected leader, sending a full tablet report...
I20260809 01:04:19.906095 26014 ts_tablet_manager.cc:1428] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0: Time spent starting tablet: real 0.018s user 0.016s sys 0.000s
I20260809 01:04:19.907013 26020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.892343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.130:41471 (local address 127.4.82.131:38769)
0809 01:04:19.894515 (+ 2172us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:19.894536 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:19.895227 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:19.895963 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:19.895968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:19.895991 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 01:04:19.896429 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.896442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.899469 (+ 3027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.899477 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.902342 (+ 2865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:19.902358 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.906311 (+ 3953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.906315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:19.906439 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.906464 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 01:04:19.906571 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 01:04:19.906704 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2043,"thread_start_us":1861,"threads_started":1}
I20260809 01:04:19.907130 26010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.892963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.82.131:38769 (local address 127.4.82.130:41471)
0809 01:04:19.893187 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:19.893192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:19.894617 (+ 1425us) server_negotiation.cc:407] Connection header received
0809 01:04:19.895598 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:19.895604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:19.895702 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:19.895804 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 01:04:19.896619 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.897752 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.902570 (+ 4818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.903726 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.903857 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.903922 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:19.906703 (+ 2781us) server_negotiation.cc:299] Negotiation successful
0809 01:04:19.906850 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 01:04:19.907754 26018 raft_consensus.cc:504] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864fb157a1f64feab8c6ce01fa846b5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36407 } } peers { permanent_uuid: "cffbf53971a84a3bbf411cb227f7961f" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44163 } } peers { permanent_uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41471 } }
I20260809 01:04:19.908252 25803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c417e0bd924783b2281b9c211d1954" candidate_uuid: "cffbf53971a84a3bbf411cb227f7961f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0" is_pre_election: true
I20260809 01:04:19.908507 26018 leader_election.cc:288] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [CANDIDATE]: Term 1 election: Requested vote from peers 864fb157a1f64feab8c6ce01fa846b5c (127.4.82.129:36407), 5a4875d2d8cc4d9eb075189ec0cda7d0 (127.4.82.130:41471)
I20260809 01:04:19.908824 25803 raft_consensus.cc:2454] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cffbf53971a84a3bbf411cb227f7961f in term 0.
I20260809 01:04:19.908931 25666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c417e0bd924783b2281b9c211d1954" candidate_uuid: "cffbf53971a84a3bbf411cb227f7961f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "864fb157a1f64feab8c6ce01fa846b5c"
I20260809 01:04:19.909114 25666 raft_consensus.cc:3037] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:19.909512 25802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c417e0bd924783b2281b9c211d1954" candidate_uuid: "cffbf53971a84a3bbf411cb227f7961f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0"
I20260809 01:04:19.909806 25802 raft_consensus.cc:3037] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:19.912389 25666 raft_consensus.cc:2454] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cffbf53971a84a3bbf411cb227f7961f in term 1.
I20260809 01:04:19.912750 25875 leader_election.cc:302] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 864fb157a1f64feab8c6ce01fa846b5c, cffbf53971a84a3bbf411cb227f7961f; no voters:
I20260809 01:04:19.913030 26018 raft_consensus.cc:2781] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:19.913075 25802 raft_consensus.cc:2454] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cffbf53971a84a3bbf411cb227f7961f in term 1.
I20260809 01:04:19.913561 26018 raft_consensus.cc:686] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [term 1 LEADER]: Becoming Leader. State: Replica: cffbf53971a84a3bbf411cb227f7961f, State: Running, Role: LEADER
I20260809 01:04:19.914017 26018 consensus_queue.cc:237] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864fb157a1f64feab8c6ce01fa846b5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36407 } } peers { permanent_uuid: "cffbf53971a84a3bbf411cb227f7961f" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44163 } } peers { permanent_uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41471 } }
I20260809 01:04:19.918274 25519 catalog_manager.cc:5495] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f reported cstate change: term changed from 0 to 1, leader changed from <none> to cffbf53971a84a3bbf411cb227f7961f (127.4.82.131). New cstate: current_term: 1 leader_uuid: "cffbf53971a84a3bbf411cb227f7961f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864fb157a1f64feab8c6ce01fa846b5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cffbf53971a84a3bbf411cb227f7961f" member_type: VOTER last_known_addr { host: "127.4.82.131" port: 44163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41471 } health_report { overall_health: UNKNOWN } } }
I20260809 01:04:19.938545 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.933567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53680 (local address 127.4.82.131:44163)
0809 01:04:19.933741 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:19.933745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:19.933840 (+ 95us) server_negotiation.cc:407] Connection header received
0809 01:04:19.934002 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:19.934006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:19.934078 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:19.934208 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:04:19.934519 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.935228 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.936670 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.937155 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.937247 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:19.937682 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:04:19.938122 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:04:19.938240 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:19.938313 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 01:04:19.938368 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 01:04:19.946651 25666 raft_consensus.cc:1261] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c [term 1 FOLLOWER]: Refusing update from remote peer cffbf53971a84a3bbf411cb227f7961f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:04:19.946661 25802 raft_consensus.cc:1261] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0 [term 1 FOLLOWER]: Refusing update from remote peer cffbf53971a84a3bbf411cb227f7961f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:04:19.947623 26022 consensus_queue.cc:1035] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a4875d2d8cc4d9eb075189ec0cda7d0" member_type: VOTER last_known_addr { host: "127.4.82.130" port: 41471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:04:19.947983 26018 consensus_queue.cc:1035] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [LEADER]: Connected to new peer: Peer: permanent_uuid: "864fb157a1f64feab8c6ce01fa846b5c" member_type: VOTER last_known_addr { host: "127.4.82.129" port: 36407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:04:19.963594 26026 mvcc.cc:205] Tried to move back new op lower bound from 7316428635931271168 to 7316428635811328000. Current Snapshot: MvccSnapshot[applied={T|T < 7316428635931271168}]
I20260809 01:04:19.969012 26027 mvcc.cc:205] Tried to move back new op lower bound from 7316428635931271168 to 7316428635811328000. Current Snapshot: MvccSnapshot[applied={T|T < 7316428635931271168}]
I20260809 01:04:19.974656 25519 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:04:19.974936 25519 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 01:04:19.978641 25519 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 542244e8190e427db7c99d398c73dc6f: Sending DeleteTablet for 3 replicas of tablet e5c417e0bd924783b2281b9c211d1954
I20260809 01:04:19.979737 25646 tablet_service.cc:1514] Processing DeleteTablet for tablet e5c417e0bd924783b2281b9c211d1954 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:19 UTC) from {username='slave'} at 127.0.0.1:57950
I20260809 01:04:19.979774 25922 tablet_service.cc:1514] Processing DeleteTablet for tablet e5c417e0bd924783b2281b9c211d1954 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:19 UTC) from {username='slave'} at 127.0.0.1:53678
I20260809 01:04:19.979877 25782 tablet_service.cc:1514] Processing DeleteTablet for tablet e5c417e0bd924783b2281b9c211d1954 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 01:04:19 UTC) from {username='slave'} at 127.0.0.1:58856
I20260809 01:04:19.981482 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 25582
I20260809 01:04:19.991808 26035 tablet_replica.cc:330] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f: stopping tablet replica
I20260809 01:04:19.992285 26035 raft_consensus.cc:2229] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [term 1 LEADER]: Raft consensus shutting down.
I20260809 01:04:19.992589 26037 tablet_replica.cc:330] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0: stopping tablet replica
I20260809 01:04:19.992928 26035 raft_consensus.cc:2258] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:19.992934 26037 raft_consensus.cc:2229] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:04:19.993435 26037 raft_consensus.cc:2258] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:19.995154 26035 ts_tablet_manager.cc:1905] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:19.996668 26037 ts_tablet_manager.cc:1905] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:20.001099 26036 tablet_replica.cc:330] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c: stopping tablet replica
I20260809 01:04:20.001675 26036 raft_consensus.cc:2229] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 01:04:20.002245 26036 raft_consensus.cc:2258] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 01:04:20.003413 26037 ts_tablet_manager.cc:1918] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:20.003463 26035 ts_tablet_manager.cc:1918] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 01:04:20.003568 26037 log.cc:1196] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-1/wal/wals/e5c417e0bd924783b2281b9c211d1954
I20260809 01:04:20.003621 26035 log.cc:1196] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f: Deleting WAL directory at /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786237144174446-4426-0/minicluster-data/ts-2/wal/wals/e5c417e0bd924783b2281b9c211d1954
I20260809 01:04:20.004181 26037 ts_tablet_manager.cc:1939] T e5c417e0bd924783b2281b9c211d1954 P 5a4875d2d8cc4d9eb075189ec0cda7d0: Deleting consensus metadata
I20260809 01:04:20.004236 26035 ts_tablet_manager.cc:1939] T e5c417e0bd924783b2281b9c211d1954 P cffbf53971a84a3bbf411cb227f7961f: Deleting consensus metadata
I20260809 01:04:20.005620 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:19.983112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57958 (local address 127.4.82.129:36407)
0809 01:04:19.983390 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:19.983398 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:19.992019 (+ 8621us) server_negotiation.cc:407] Connection header received
0809 01:04:19.992347 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:19.992352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:19.992474 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:19.992555 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:19.993231 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.994592 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.996417 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:19.997031 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:19.997190 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:20.003060 (+ 5870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:20.003074 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:20.003092 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:20.003921 (+ 829us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:20.004005 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:20.004040 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:20.004205 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:20.005090 (+ 885us) server_negotiation.cc:299] Negotiation successful
0809 01:04:20.005219 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 01:04:20.006119 25505 catalog_manager.cc:4841] TS cffbf53971a84a3bbf411cb227f7961f (127.4.82.131:44163): tablet e5c417e0bd924783b2281b9c211d1954 (table test-table [id=244d02f5d49e475ba03bc9c17464d977]) successfully deleted
I20260809 01:04:20.006704 25709 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:20.007615 25503 catalog_manager.cc:4841] TS 5a4875d2d8cc4d9eb075189ec0cda7d0 (127.4.82.130:41471): tablet e5c417e0bd924783b2281b9c211d1954 (table test-table [id=244d02f5d49e475ba03bc9c17464d977]) successfully deleted
I20260809 01:04:20.007570 26036 ts_tablet_manager.cc:1905] T e5c417e0bd924783b2281b9c211d1954 P 864fb157a1f64feab8c6ce01fa846b5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 01:04:20.096767 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 25582
I20260809 01:04:20.117817 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 25718
W20260809 01:04:20.117822 25505 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.82.129:36407 (error 108)
I20260809 01:04:20.120194 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:20.119600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36407 (local address 127.0.0.1:57968)
0809 01:04:20.119868 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:20.119989 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260809 01:04:20.120478 25505 catalog_manager.cc:4584] TS 864fb157a1f64feab8c6ce01fa846b5c (127.4.82.129:36407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5c417e0bd924783b2281b9c211d1954: Network error: Client connection negotiation failed: client connection to 127.4.82.129:36407: connect: Connection refused (error 111)
I20260809 01:04:20.126155 26010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:20.119009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58870 (local address 127.4.82.130:41471)
0809 01:04:20.119251 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:20.119259 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:20.119327 (+ 68us) server_negotiation.cc:407] Connection header received
0809 01:04:20.119454 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:20.119461 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:20.119792 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:20.120693 (+ 901us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:20.120851 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:20.121540 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:20.122792 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:20.123208 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:20.123626 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:20.124689 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:20.124699 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:20.124713 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:20.125264 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:20.125309 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:20.125328 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:20.125457 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:20.125593 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 01:04:20.125700 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 01:04:20.126788 25846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:20.174213 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:20.173647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36407 (local address 127.0.0.1:57974)
0809 01:04:20.173898 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:20.174011 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 01:04:20.175470 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:20.175001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36407 (local address 127.0.0.1:57978)
0809 01:04:20.175187 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:20.175275 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 01:04:20.218767 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 25718
I20260809 01:04:20.236454 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:20.235852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36407 (local address 127.0.0.1:57992)
0809 01:04:20.236124 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:20.236246 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 01:04:20.237733 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:20.237274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36407 (local address 127.0.0.1:58008)
0809 01:04:20.237461 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:20.237558 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 01:04:20.238440 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 25856
I20260809 01:04:20.246767 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:20.239194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53696 (local address 127.4.82.131:44163)
0809 01:04:20.239387 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:20.239392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:20.239428 (+ 36us) server_negotiation.cc:407] Connection header received
0809 01:04:20.239667 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:20.239671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:20.239778 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:20.239860 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:20.240438 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:20.241259 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:20.243124 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:20.243673 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:20.243766 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:20.245096 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:20.245110 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:20.245137 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:20.245940 (+ 803us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:20.246005 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:20.246029 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:20.246127 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:20.246321 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 01:04:20.246419 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 01:04:20.247741 25986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:20.318519 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:20.317956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36407 (local address 127.0.0.1:58012)
0809 01:04:20.318214 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:20.318328 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 01:04:20.319703 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:20.319197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.129:36407 (local address 127.0.0.1:58020)
0809 01:04:20.319434 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:20.319530 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.129:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260809 01:04:20.369179 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 25856
I20260809 01:04:20.390051 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 25486
I20260809 01:04:20.391093 25552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:20.511101 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 25486
2026-08-09T01:04:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2789 ms)
[----------] 6 tests from EncryptionPolicyTest (21924 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:20 dist-test-slave-07c2 krb5kdc[26048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:20 dist-test-slave-07c2 krb5kdc[26048](info): set up 2 sockets
Aug 09 01:04:20 dist-test-slave-07c2 krb5kdc[26048](info): commencing operation
krb5kdc: starting...
W20260809 01:04:21.683214 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:21 dist-test-slave-07c2 krb5kdc[26048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 01:04:21.970796 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33127
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:22.308524 26061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:22.308866 26061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:22.309046 26061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:22.316393 26061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:22.316531 26061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:22.316619 26061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:22.316730 26061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:22.316797 26061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:22.326557 26061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33127
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:22.327883 26061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:22.329198 26061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:22.340776 26069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:22.340867 26067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:22.341089 26066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:22.341845 26061 server_base.cc:1034] running on GCE node
I20260809 01:04:22.342567 26061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:22.343969 26061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:22.345209 26061 hybrid_clock.cc:648] HybridClock initialized: now 1786237462345171 us; error 44 us; skew 500 ppm
I20260809 01:04:22.345547 26061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:22 dist-test-slave-07c2 krb5kdc[26048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237462, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:22.348641 26061 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 01:04:22.350106 26061 webserver.cc:460] Webserver started at http://127.0.0.1:33303/ using document root <none> and password file <none>
I20260809 01:04:22.350790 26061 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:22.350940 26061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:22.351254 26061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:22.354187 26061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "8755759b68b7415b8ea846b92cbe9a83"
format_stamp: "Formatted at 2026-08-09 01:04:22 on dist-test-slave-07c2"
server_key: "b858a51aadfc557da997ab7f0b47592b"
server_key_iv: "d75b20418606338291e6dba21bfadd34"
server_key_version: "encryptionkey@0"
I20260809 01:04:22.354946 26061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "8755759b68b7415b8ea846b92cbe9a83"
format_stamp: "Formatted at 2026-08-09 01:04:22 on dist-test-slave-07c2"
server_key: "b858a51aadfc557da997ab7f0b47592b"
server_key_iv: "d75b20418606338291e6dba21bfadd34"
server_key_version: "encryptionkey@0"
I20260809 01:04:22.360008 26061 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 01:04:22.363385 26076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:22.364567 26061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 01:04:22.364758 26061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "8755759b68b7415b8ea846b92cbe9a83"
format_stamp: "Formatted at 2026-08-09 01:04:22 on dist-test-slave-07c2"
server_key: "b858a51aadfc557da997ab7f0b47592b"
server_key_iv: "d75b20418606338291e6dba21bfadd34"
server_key_version: "encryptionkey@0"
I20260809 01:04:22.364951 26061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:22.386021 26061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:22.389752 26061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:22.390026 26061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:22.408595 26061 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33127
I20260809 01:04:22.408629 26134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33127 every 8 connection(s)
I20260809 01:04:22.409930 26061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:22.413602 26135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:22.418561 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 26061
I20260809 01:04:22.418780 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:22.419658 4426 external_mini_cluster.cc:1428] Setting key 92728f3087d67f5783bd8155216d7301
I20260809 01:04:22.425387 26135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83: Bootstrap starting.
I20260809 01:04:22.429028 26135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:22.430279 26135 log.cc:824] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:22.433139 26135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83: No bootstrap required, opened a new log
Aug 09 01:04:22 dist-test-slave-07c2 krb5kdc[26048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 01:04:22.442626 26135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8755759b68b7415b8ea846b92cbe9a83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33127 } }
I20260809 01:04:22.442873 26138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:22.421625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37173 (local address 127.0.0.1:33127)
0809 01:04:22.422476 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:22.422487 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:22.422535 (+ 48us) server_negotiation.cc:407] Connection header received
0809 01:04:22.424315 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:22.424336 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:22.424680 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:22.424982 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:22.426789 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:22.428212 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:22.430116 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:22.430971 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:22.431180 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:22.435707 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:22.435714 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:22.435727 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:22.435766 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:22.438847 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:22.439797 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:22.439803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:22.439809 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:22.439981 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:22.440436 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:22.440440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:22.440442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:22.440916 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:22.441178 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:22.441632 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 01:04:22.441926 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":402,"threads_started":1}
I20260809 01:04:22.442976 26135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:22.443066 26135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8755759b68b7415b8ea846b92cbe9a83, State: Initialized, Role: FOLLOWER
I20260809 01:04:22.443650 26135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8755759b68b7415b8ea846b92cbe9a83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33127 } }
I20260809 01:04:22.443851 26135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:22.443962 26135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:22.444079 26135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:22.446558 26135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8755759b68b7415b8ea846b92cbe9a83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33127 } }
I20260809 01:04:22.446781 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:22.447072 26135 leader_election.cc:302] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8755759b68b7415b8ea846b92cbe9a83; no voters:
I20260809 01:04:22.447695 26135 leader_election.cc:288] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:22.447798 26140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:22.449270 26140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [term 1 LEADER]: Becoming Leader. State: Replica: 8755759b68b7415b8ea846b92cbe9a83, State: Running, Role: LEADER
I20260809 01:04:22.449806 26135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:22.449781 26140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8755759b68b7415b8ea846b92cbe9a83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33127 } }
I20260809 01:04:22.452642 26142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8755759b68b7415b8ea846b92cbe9a83. Latest consensus state: current_term: 1 leader_uuid: "8755759b68b7415b8ea846b92cbe9a83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8755759b68b7415b8ea846b92cbe9a83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33127 } } }
I20260809 01:04:22.453029 26142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:22.453548 26141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8755759b68b7415b8ea846b92cbe9a83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8755759b68b7415b8ea846b92cbe9a83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33127 } } }
I20260809 01:04:22.453863 26141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:22.456710 26150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:22.462100 26150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:22.476929 26138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:22.456287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42019 (local address 127.0.0.1:33127)
0809 01:04:22.456445 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:22.456450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:22.457789 (+ 1339us) server_negotiation.cc:407] Connection header received
0809 01:04:22.458057 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:22.458062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:22.458140 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:22.458214 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:22.460467 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:22.461652 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:22.463641 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:22.464316 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:22.464438 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:22.467809 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:22.467817 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:22.467823 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:22.467864 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:22.472210 (+ 4346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:22.473521 (+ 1311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:22.473530 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:22.473536 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:22.473756 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:22.474480 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:22.474488 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:22.474494 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:22.474994 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:22.475351 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:22.476534 (+ 1183us) server_negotiation.cc:299] Negotiation successful
0809 01:04:22.476677 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 01:04:22.478484 26150 catalog_manager.cc:1347] Generated new cluster ID: 5653fcede3024a08999b3e50cf86ca65
I20260809 01:04:22.478605 26150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:22.496943 26150 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:22.498253 26150 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:22.509331 26150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8755759b68b7415b8ea846b92cbe9a83: Generated new TSK 0
I20260809 01:04:22.510206 26150 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:22.517789 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 26061
I20260809 01:04:22.519454 26130 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:22.610761 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 26061
W20260809 01:04:22.630566 26045 connection.cc:535] client connection to 127.0.0.1:33127 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33127): Connection reset by peer (error 104)
2026-08-09T01:04:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2081 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:22 dist-test-slave-07c2 krb5kdc[26168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:22 dist-test-slave-07c2 krb5kdc[26168](info): set up 2 sockets
Aug 09 01:04:22 dist-test-slave-07c2 krb5kdc[26168](info): commencing operation
krb5kdc: starting...
W20260809 01:04:23.759188 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:23 dist-test-slave-07c2 krb5kdc[26168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 01:04:24.047602 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37059
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:24.379629 26182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:24.379891 26182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:24.380079 26182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:24.387022 26182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:24.387159 26182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:24.387250 26182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:24.387387 26182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:24.387475 26182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:24.397243 26182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37059
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:24.398458 26182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:24.399758 26182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:24.411790 26188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:24.411885 26192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:24.411907 26190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:24.412679 26182 server_base.cc:1034] running on GCE node
I20260809 01:04:24.413481 26182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:24.414837 26182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:24.416059 26182 hybrid_clock.cc:648] HybridClock initialized: now 1786237464416024 us; error 30 us; skew 500 ppm
I20260809 01:04:24.416412 26182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:24 dist-test-slave-07c2 krb5kdc[26168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237464, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:24.419579 26182 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 01:04:24.421073 26182 webserver.cc:460] Webserver started at http://127.0.0.1:38503/ using document root <none> and password file <none>
I20260809 01:04:24.421757 26182 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:24.421912 26182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:24.422230 26182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:24.425763 26182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "4c01bd93385d4d86830b249bf66c6dfd"
format_stamp: "Formatted at 2026-08-09 01:04:24 on dist-test-slave-07c2"
server_key: "f38757bb4a187bd8034bdcd04bb8dbe5"
server_key_iv: "b643d093a3c76ed83b0972e13881f0a5"
server_key_version: "encryptionkey@0"
I20260809 01:04:24.426874 26182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "4c01bd93385d4d86830b249bf66c6dfd"
format_stamp: "Formatted at 2026-08-09 01:04:24 on dist-test-slave-07c2"
server_key: "f38757bb4a187bd8034bdcd04bb8dbe5"
server_key_iv: "b643d093a3c76ed83b0972e13881f0a5"
server_key_version: "encryptionkey@0"
I20260809 01:04:24.431907 26182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 01:04:24.435240 26201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:24.436463 26182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 01:04:24.436642 26182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "4c01bd93385d4d86830b249bf66c6dfd"
format_stamp: "Formatted at 2026-08-09 01:04:24 on dist-test-slave-07c2"
server_key: "f38757bb4a187bd8034bdcd04bb8dbe5"
server_key_iv: "b643d093a3c76ed83b0972e13881f0a5"
server_key_version: "encryptionkey@0"
I20260809 01:04:24.436831 26182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:24.452497 26182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:24.456203 26182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:24.456473 26182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:24.475392 26182 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37059
I20260809 01:04:24.475430 26256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37059 every 8 connection(s)
I20260809 01:04:24.476730 26182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:24.480258 26257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:24.486348 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 26182
I20260809 01:04:24.486547 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:24.487134 4426 external_mini_cluster.cc:1428] Setting key d9ad7d91603251f22961f6fa6192f1cf
I20260809 01:04:24.492020 26257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd: Bootstrap starting.
I20260809 01:04:24.494998 26257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:24.496069 26257 log.cc:824] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:24.499199 26257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd: No bootstrap required, opened a new log
I20260809 01:04:24.500205 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:24.488974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41235 (local address 127.0.0.1:37059)
0809 01:04:24.489799 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:24.489810 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:24.489850 (+ 40us) server_negotiation.cc:407] Connection header received
0809 01:04:24.491570 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:24.491591 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:24.491965 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:24.492381 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:24.492915 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:24.494227 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:24.495688 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:24.496398 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:24.496590 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:24.497922 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:24.497929 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:24.497942 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:24.498450 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:24.498521 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:24.498545 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:24.498673 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:24.498974 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 01:04:24.499166 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":356,"threads_started":1}
I20260809 01:04:24.504053 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:24.507594 26257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c01bd93385d4d86830b249bf66c6dfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37059 } }
I20260809 01:04:24.507967 26257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:24.508095 26257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c01bd93385d4d86830b249bf66c6dfd, State: Initialized, Role: FOLLOWER
I20260809 01:04:24.508956 26257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c01bd93385d4d86830b249bf66c6dfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37059 } }
I20260809 01:04:24.509255 26257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:24.509439 26257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:24.509640 26257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:24.512055 26257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c01bd93385d4d86830b249bf66c6dfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37059 } }
I20260809 01:04:24.512583 26257 leader_election.cc:302] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c01bd93385d4d86830b249bf66c6dfd; no voters:
I20260809 01:04:24.513299 26257 leader_election.cc:288] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:24.513401 26267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:24.515200 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:24.508526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50303 (local address 127.0.0.1:37059)
0809 01:04:24.508699 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:24.508704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:24.509180 (+ 476us) server_negotiation.cc:407] Connection header received
0809 01:04:24.509387 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:24.509393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:24.509465 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:24.509569 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:24.510082 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:24.510893 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:24.512295 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:24.512787 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:24.512905 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:24.514241 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:24.514246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:24.514251 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:24.514270 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:24.514316 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:24.514360 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:24.514436 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:24.514804 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 01:04:24.514860 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 01:04:24.515988 26257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:24.519580 26267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [term 1 LEADER]: Becoming Leader. State: Replica: 4c01bd93385d4d86830b249bf66c6dfd, State: Running, Role: LEADER
I20260809 01:04:24.520325 26267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c01bd93385d4d86830b249bf66c6dfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37059 } }
I20260809 01:04:24.523715 26268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c01bd93385d4d86830b249bf66c6dfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c01bd93385d4d86830b249bf66c6dfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37059 } } }
I20260809 01:04:24.524147 26268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:24.524827 26269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c01bd93385d4d86830b249bf66c6dfd. Latest consensus state: current_term: 1 leader_uuid: "4c01bd93385d4d86830b249bf66c6dfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c01bd93385d4d86830b249bf66c6dfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37059 } } }
I20260809 01:04:24.525697 26269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:24.527961 26278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:24.532902 26278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:24.541771 26278 catalog_manager.cc:1347] Generated new cluster ID: ac8ee0820ede4699a27c4e86b781d6d1
I20260809 01:04:24.541890 26278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:24.570744 26278 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:24.572674 26278 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:24.583389 26278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c01bd93385d4d86830b249bf66c6dfd: Generated new TSK 0
I20260809 01:04:24.584439 26278 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:24.593686 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 26182
W20260809 01:04:24.595100 26252 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41235
W20260809 01:04:24.595870 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 26182: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 01:04:24.596067 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 26182
W20260809 01:04:24.616878 26166 connection.cc:535] client connection to 127.0.0.1:37059 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37059): Connection reset by peer (error 104)
2026-08-09T01:04:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1985 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:24 dist-test-slave-07c2 krb5kdc[26290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:24 dist-test-slave-07c2 krb5kdc[26290](info): set up 2 sockets
Aug 09 01:04:24 dist-test-slave-07c2 krb5kdc[26290](info): commencing operation
krb5kdc: starting...
W20260809 01:04:25.750444 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:25 dist-test-slave-07c2 krb5kdc[26290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 01:04:26.034570 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38357
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:26.362476 26304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:26.362764 26304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:26.362952 26304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:26.369972 26304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:26.370066 26304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:26.370111 26304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:26.370203 26304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:26.370271 26304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:26.380107 26304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38357
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:26.381309 26304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:26.382573 26304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:26.394088 26313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:26.394240 26309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:26.394196 26310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:26.395048 26304 server_base.cc:1034] running on GCE node
I20260809 01:04:26.395795 26304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:26.397053 26304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:26.398257 26304 hybrid_clock.cc:648] HybridClock initialized: now 1786237466398224 us; error 25 us; skew 500 ppm
I20260809 01:04:26.398586 26304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:26 dist-test-slave-07c2 krb5kdc[26290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237466, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:26.401571 26304 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 01:04:26.403044 26304 webserver.cc:460] Webserver started at http://127.0.0.1:43729/ using document root <none> and password file <none>
I20260809 01:04:26.403762 26304 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:26.403909 26304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:26.404208 26304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:26.407018 26304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "0bdb29e9f61043a7b2e8b0583039ba5a"
format_stamp: "Formatted at 2026-08-09 01:04:26 on dist-test-slave-07c2"
server_key: "a569a4eb2afb18a3f1076d1592f58d62"
server_key_iv: "a8c946bce02a4ddce9344787edd379b4"
server_key_version: "encryptionkey@0"
I20260809 01:04:26.407869 26304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "0bdb29e9f61043a7b2e8b0583039ba5a"
format_stamp: "Formatted at 2026-08-09 01:04:26 on dist-test-slave-07c2"
server_key: "a569a4eb2afb18a3f1076d1592f58d62"
server_key_iv: "a8c946bce02a4ddce9344787edd379b4"
server_key_version: "encryptionkey@0"
I20260809 01:04:26.412743 26304 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 01:04:26.416015 26321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:26.417191 26304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 01:04:26.417378 26304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "0bdb29e9f61043a7b2e8b0583039ba5a"
format_stamp: "Formatted at 2026-08-09 01:04:26 on dist-test-slave-07c2"
server_key: "a569a4eb2afb18a3f1076d1592f58d62"
server_key_iv: "a8c946bce02a4ddce9344787edd379b4"
server_key_version: "encryptionkey@0"
I20260809 01:04:26.417577 26304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:26.423100 26304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:26.426873 26304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:26.427160 26304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:26.446821 26304 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38357
I20260809 01:04:26.446906 26377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38357 every 8 connection(s)
I20260809 01:04:26.448310 26304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:26.451893 26378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:26.453354 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 26304
I20260809 01:04:26.453547 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:26.454131 4426 external_mini_cluster.cc:1428] Setting key 8f438ec100d13289db2d473fb8dfa748
I20260809 01:04:26.461516 26381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:26.455919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38657 (local address 127.0.0.1:38357)
0809 01:04:26.456731 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:26.456742 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:26.456786 (+ 44us) server_negotiation.cc:407] Connection header received
0809 01:04:26.458334 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:26.458356 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:26.458672 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:26.458990 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:26.460046 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:26.460052 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:26.460063 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:26.460077 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:26.460134 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:26.460152 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:26.460326 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:26.460463 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 01:04:26.460647 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":384,"threads_started":1}
I20260809 01:04:26.465173 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:26.465653 26378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a: Bootstrap starting.
I20260809 01:04:26.468614 26378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:26.469933 26378 log.cc:824] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:26.472241 26381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:26.469583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55199 (local address 127.0.0.1:38357)
0809 01:04:26.469737 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:26.469741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:26.470221 (+ 480us) server_negotiation.cc:407] Connection header received
0809 01:04:26.470405 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:26.470414 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:26.470463 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:26.470563 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:26.471594 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:26.471598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:26.471601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:26.471616 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:26.471652 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:26.471666 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:26.471764 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:26.471909 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 01:04:26.471949 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 01:04:26.473038 26378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a: No bootstrap required, opened a new log
I20260809 01:04:26.479908 26378 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bdb29e9f61043a7b2e8b0583039ba5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38357 } }
I20260809 01:04:26.480268 26378 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:26.480376 26378 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdb29e9f61043a7b2e8b0583039ba5a, State: Initialized, Role: FOLLOWER
I20260809 01:04:26.480903 26378 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bdb29e9f61043a7b2e8b0583039ba5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38357 } }
I20260809 01:04:26.481153 26378 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:26.481303 26378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:26.481473 26378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:26.483616 26378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bdb29e9f61043a7b2e8b0583039ba5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38357 } }
I20260809 01:04:26.484089 26378 leader_election.cc:302] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bdb29e9f61043a7b2e8b0583039ba5a; no voters:
I20260809 01:04:26.484685 26378 leader_election.cc:288] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:26.484814 26388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:26.486225 26388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [term 1 LEADER]: Becoming Leader. State: Replica: 0bdb29e9f61043a7b2e8b0583039ba5a, State: Running, Role: LEADER
I20260809 01:04:26.486707 26388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bdb29e9f61043a7b2e8b0583039ba5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38357 } }
I20260809 01:04:26.486847 26378 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:26.489768 26389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bdb29e9f61043a7b2e8b0583039ba5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bdb29e9f61043a7b2e8b0583039ba5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38357 } } }
I20260809 01:04:26.489641 26391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bdb29e9f61043a7b2e8b0583039ba5a. Latest consensus state: current_term: 1 leader_uuid: "0bdb29e9f61043a7b2e8b0583039ba5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bdb29e9f61043a7b2e8b0583039ba5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38357 } } }
I20260809 01:04:26.490124 26391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:26.490125 26389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:26.491071 26399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:26.495378 26399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:26.503759 26399 catalog_manager.cc:1347] Generated new cluster ID: f298587b92314c9c823f241a3d22f23c
I20260809 01:04:26.503868 26399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:26.514881 26399 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:26.515362 26399 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:26.532370 26399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bdb29e9f61043a7b2e8b0583039ba5a: Generated new TSK 0
I20260809 01:04:26.533200 26399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:26.545552 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 26304
W20260809 01:04:26.546784 26372 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38657
W20260809 01:04:26.547571 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 26304: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 01:04:26.547787 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 26304
2026-08-09T01:04:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1950 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:26 dist-test-slave-07c2 krb5kdc[26411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:26 dist-test-slave-07c2 krb5kdc[26411](info): set up 2 sockets
Aug 09 01:04:26 dist-test-slave-07c2 krb5kdc[26411](info): commencing operation
krb5kdc: starting...
W20260809 01:04:27.698881 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:27 dist-test-slave-07c2 krb5kdc[26411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 01:04:28.010912 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36557
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:28.356846 26424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:28.357218 26424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:28.357379 26424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:28.365157 26424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:28.365339 26424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:28.365446 26424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:28.365563 26424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:28.365641 26424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:28.376300 26424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:28.377754 26424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:28.379180 26424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:28.391626 26433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:28.391709 26431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:28.391706 26430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:28.396682 26424 server_base.cc:1034] running on GCE node
I20260809 01:04:28.397477 26424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:28.398890 26424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:28.400115 26424 hybrid_clock.cc:648] HybridClock initialized: now 1786237468400082 us; error 49 us; skew 500 ppm
I20260809 01:04:28.400478 26424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:28 dist-test-slave-07c2 krb5kdc[26411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237468, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:28.403575 26424 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 01:04:28.405112 26424 webserver.cc:460] Webserver started at http://127.0.0.1:40705/ using document root <none> and password file <none>
I20260809 01:04:28.405830 26424 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:28.405982 26424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:28.406291 26424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:28.409291 26424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "34ffa0c799ea4f9e97b2f205b4bbf2a7"
format_stamp: "Formatted at 2026-08-09 01:04:28 on dist-test-slave-07c2"
server_key: "10f72e0c9ea29655225b2de1088561df"
server_key_iv: "a0d5334965eb450d54b2d20c3c8c9be9"
server_key_version: "encryptionkey@0"
I20260809 01:04:28.410094 26424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "34ffa0c799ea4f9e97b2f205b4bbf2a7"
format_stamp: "Formatted at 2026-08-09 01:04:28 on dist-test-slave-07c2"
server_key: "10f72e0c9ea29655225b2de1088561df"
server_key_iv: "a0d5334965eb450d54b2d20c3c8c9be9"
server_key_version: "encryptionkey@0"
I20260809 01:04:28.415089 26424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 01:04:28.418531 26441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:28.419812 26424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:04:28.419994 26424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "34ffa0c799ea4f9e97b2f205b4bbf2a7"
format_stamp: "Formatted at 2026-08-09 01:04:28 on dist-test-slave-07c2"
server_key: "10f72e0c9ea29655225b2de1088561df"
server_key_iv: "a0d5334965eb450d54b2d20c3c8c9be9"
server_key_version: "encryptionkey@0"
I20260809 01:04:28.420187 26424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:28.437554 26424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:28.442378 26424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:28.442655 26424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:28.461920 26424 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36557
I20260809 01:04:28.461962 26500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36557 every 8 connection(s)
I20260809 01:04:28.463369 26424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:28.466964 26501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:28.471154 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 26424
I20260809 01:04:28.471369 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:28.471930 4426 external_mini_cluster.cc:1428] Setting key 3add0426b488bc7f087107cb22af4bf5
I20260809 01:04:28.480324 26501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7: Bootstrap starting.
I20260809 01:04:28.484129 26501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:28.485482 26501 log.cc:824] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7: Log is configured to *not* fsync() on all Append() calls
Aug 09 01:04:28 dist-test-slave-07c2 krb5kdc[26411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 01:04:28.488521 26501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7: No bootstrap required, opened a new log
I20260809 01:04:28.495587 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:28.473685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44389 (local address 127.0.0.1:36557)
0809 01:04:28.474424 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:28.474436 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:28.474478 (+ 42us) server_negotiation.cc:407] Connection header received
0809 01:04:28.476115 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:28.476135 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:28.476506 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:28.476836 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:28.478769 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:28.480132 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:28.481972 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:28.482634 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:28.482771 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:28.488169 (+ 5398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:28.488177 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:28.488191 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:28.488226 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:28.491454 (+ 3228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:28.492173 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:28.492181 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:28.492196 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:28.492360 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:28.492783 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:28.492786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:28.492788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:28.493301 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:28.493623 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:28.494070 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 01:04:28.494505 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":360,"threads_started":1}
I20260809 01:04:28.496448 26501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ffa0c799ea4f9e97b2f205b4bbf2a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36557 } }
I20260809 01:04:28.496843 26501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:28.496974 26501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34ffa0c799ea4f9e97b2f205b4bbf2a7, State: Initialized, Role: FOLLOWER
I20260809 01:04:28.497536 26501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ffa0c799ea4f9e97b2f205b4bbf2a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36557 } }
I20260809 01:04:28.497819 26501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:28.497984 26501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:28.498178 26501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:28.499723 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:28.501433 26501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ffa0c799ea4f9e97b2f205b4bbf2a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36557 } }
I20260809 01:04:28.501997 26501 leader_election.cc:302] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34ffa0c799ea4f9e97b2f205b4bbf2a7; no voters:
I20260809 01:04:28.502678 26501 leader_election.cc:288] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:28.502795 26507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:28.505218 26501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:28.511984 26507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [term 1 LEADER]: Becoming Leader. State: Replica: 34ffa0c799ea4f9e97b2f205b4bbf2a7, State: Running, Role: LEADER
I20260809 01:04:28.512837 26507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ffa0c799ea4f9e97b2f205b4bbf2a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36557 } }
I20260809 01:04:28.516782 26508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34ffa0c799ea4f9e97b2f205b4bbf2a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ffa0c799ea4f9e97b2f205b4bbf2a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36557 } } }
I20260809 01:04:28.517445 26508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:28.517086 26510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34ffa0c799ea4f9e97b2f205b4bbf2a7. Latest consensus state: current_term: 1 leader_uuid: "34ffa0c799ea4f9e97b2f205b4bbf2a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ffa0c799ea4f9e97b2f205b4bbf2a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36557 } } }
I20260809 01:04:28.518590 26510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:28.519798 26523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:28.525462 26523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:28.541021 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:28.514886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55137 (local address 127.0.0.1:36557)
0809 01:04:28.515110 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:28.515120 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:28.518611 (+ 3491us) server_negotiation.cc:407] Connection header received
0809 01:04:28.518931 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:28.518938 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:28.519010 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:28.519079 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:28.521354 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:28.522875 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:28.525516 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:28.526487 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:28.526647 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:28.530831 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:28.530841 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:28.530848 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:28.530904 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:28.535841 (+ 4937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:28.537013 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:28.537021 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:28.537027 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:28.537205 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:28.537799 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:28.537806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:28.537816 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:28.538355 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:28.538658 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:28.540154 (+ 1496us) server_negotiation.cc:299] Negotiation successful
0809 01:04:28.540476 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 01:04:28.541517 26523 catalog_manager.cc:1347] Generated new cluster ID: 9781d946d6224b2d92cfbb4e98c7fc1b
I20260809 01:04:28.541637 26523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:28.566661 26523 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:28.568152 26523 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:28.588475 26523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34ffa0c799ea4f9e97b2f205b4bbf2a7: Generated new TSK 0
I20260809 01:04:28.589376 26523 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:28.599762 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 26424
I20260809 01:04:28.601586 26496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:28.694963 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 26424
2026-08-09T01:04:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2147 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:28 dist-test-slave-07c2 krb5kdc[26535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:28 dist-test-slave-07c2 krb5kdc[26535](info): set up 2 sockets
Aug 09 01:04:28 dist-test-slave-07c2 krb5kdc[26535](info): commencing operation
krb5kdc: starting...
W20260809 01:04:29.850494 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:30 dist-test-slave-07c2 krb5kdc[26535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 01:04:30.150024 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37231
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:30.489394 26550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:30.489676 26550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:30.489849 26550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:30.496990 26550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:30.497128 26550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:30.497229 26550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:30.497339 26550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:30.497421 26550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:30.507148 26550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:30.508400 26550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:30.509702 26550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:30.521265 26556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:30.521337 26557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:30.521457 26559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:30.522218 26550 server_base.cc:1034] running on GCE node
I20260809 01:04:30.523058 26550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:30.524507 26550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:30.525748 26550 hybrid_clock.cc:648] HybridClock initialized: now 1786237470525693 us; error 47 us; skew 500 ppm
I20260809 01:04:30.526096 26550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:30 dist-test-slave-07c2 krb5kdc[26535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237470, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:30.529302 26550 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 01:04:30.530680 26550 webserver.cc:460] Webserver started at http://127.0.0.1:44571/ using document root <none> and password file <none>
I20260809 01:04:30.531375 26550 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:30.531490 26550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:30.531744 26550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:30.534584 26550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "276cdbe016bb49908f9131275435f553"
format_stamp: "Formatted at 2026-08-09 01:04:30 on dist-test-slave-07c2"
server_key: "fefb97ccac8b3f2fa31d80c62f0eede6"
server_key_iv: "08a967421bb18ef2c0f9b4b85a53998e"
server_key_version: "encryptionkey@0"
I20260809 01:04:30.535368 26550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "276cdbe016bb49908f9131275435f553"
format_stamp: "Formatted at 2026-08-09 01:04:30 on dist-test-slave-07c2"
server_key: "fefb97ccac8b3f2fa31d80c62f0eede6"
server_key_iv: "08a967421bb18ef2c0f9b4b85a53998e"
server_key_version: "encryptionkey@0"
I20260809 01:04:30.540745 26550 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 01:04:30.545863 26567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:30.547771 26550 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 01:04:30.547966 26550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "276cdbe016bb49908f9131275435f553"
format_stamp: "Formatted at 2026-08-09 01:04:30 on dist-test-slave-07c2"
server_key: "fefb97ccac8b3f2fa31d80c62f0eede6"
server_key_iv: "08a967421bb18ef2c0f9b4b85a53998e"
server_key_version: "encryptionkey@0"
I20260809 01:04:30.548225 26550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:30.576588 26550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:30.588440 26550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:30.588827 26550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:30.608426 26550 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37231
I20260809 01:04:30.608456 26621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37231 every 8 connection(s)
I20260809 01:04:30.609890 26550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:30.613507 26622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:30.619660 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 26550
I20260809 01:04:30.619824 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:30.620260 4426 external_mini_cluster.cc:1428] Setting key d4d1bde686a115058937aaec0524c7cc
I20260809 01:04:30.624226 26622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553: Bootstrap starting.
I20260809 01:04:30.627698 26622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:30.629012 26622 log.cc:824] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:30.632232 26622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553: No bootstrap required, opened a new log
I20260809 01:04:30.634241 26625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:30.622146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42445 (local address 127.0.0.1:37231)
0809 01:04:30.622937 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:30.622948 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:30.622989 (+ 41us) server_negotiation.cc:407] Connection header received
0809 01:04:30.624805 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:30.624827 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:30.625175 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:30.625472 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:30.626155 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:30.627501 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:30.629401 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:30.630089 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:30.630225 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:30.631491 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:30.631501 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:30.631518 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:30.632213 (+ 695us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:30.632309 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:30.632341 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:30.632454 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:30.632647 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 01:04:30.632953 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":359,"threads_started":1}
I20260809 01:04:30.637897 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:30.640969 26622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276cdbe016bb49908f9131275435f553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37231 } }
I20260809 01:04:30.641335 26622 raft_consensus.cc:374] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:30.641459 26622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 276cdbe016bb49908f9131275435f553, State: Initialized, Role: FOLLOWER
I20260809 01:04:30.642028 26622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276cdbe016bb49908f9131275435f553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37231 } }
I20260809 01:04:30.642349 26622 raft_consensus.cc:388] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:30.642522 26622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:30.642781 26622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:30.645856 26622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276cdbe016bb49908f9131275435f553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37231 } }
I20260809 01:04:30.646427 26622 leader_election.cc:302] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 276cdbe016bb49908f9131275435f553; no voters:
I20260809 01:04:30.647150 26622 leader_election.cc:288] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:30.647324 26633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:30.648708 26633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [term 1 LEADER]: Becoming Leader. State: Replica: 276cdbe016bb49908f9131275435f553, State: Running, Role: LEADER
I20260809 01:04:30.649320 26633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276cdbe016bb49908f9131275435f553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37231 } }
I20260809 01:04:30.649705 26622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:30.649705 26625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:30.642403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40181 (local address 127.0.0.1:37231)
0809 01:04:30.642620 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:30.642628 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:30.643034 (+ 406us) server_negotiation.cc:407] Connection header received
0809 01:04:30.643221 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:30.643225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:30.643323 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:30.643431 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:30.644015 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:30.644702 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:30.646514 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:30.647024 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:30.647195 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:30.648568 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:30.648577 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:30.648583 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:30.648608 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:30.648664 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:30.648699 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:30.648809 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:30.649283 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 01:04:30.649376 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 01:04:30.652345 26635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 276cdbe016bb49908f9131275435f553. Latest consensus state: current_term: 1 leader_uuid: "276cdbe016bb49908f9131275435f553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276cdbe016bb49908f9131275435f553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37231 } } }
I20260809 01:04:30.652844 26635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:30.653447 26634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "276cdbe016bb49908f9131275435f553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "276cdbe016bb49908f9131275435f553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37231 } } }
I20260809 01:04:30.653777 26634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:30.661640 26641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:30.667197 26641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:30.676970 26641 catalog_manager.cc:1347] Generated new cluster ID: 68f66f998ba94dd392b5bff7d7c1ba3c
I20260809 01:04:30.677098 26641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:30.688483 26641 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:30.689654 26641 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:30.703246 26641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 276cdbe016bb49908f9131275435f553: Generated new TSK 0
I20260809 01:04:30.704005 26641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:30.714757 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 26550
W20260809 01:04:30.715801 26617 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42445
W20260809 01:04:30.716519 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 26550: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 01:04:30.716673 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 26550
2026-08-09T01:04:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2020 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:30 dist-test-slave-07c2 krb5kdc[26657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:30 dist-test-slave-07c2 krb5kdc[26657](info): set up 2 sockets
Aug 09 01:04:30 dist-test-slave-07c2 krb5kdc[26657](info): commencing operation
krb5kdc: starting...
W20260809 01:04:31.863216 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.007s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:32 dist-test-slave-07c2 krb5kdc[26657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 01:04:32.162189 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37095
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37095
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:32.497416 26670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:32.497713 26670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:32.497902 26670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:32.505633 26670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:32.505754 26670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:32.505872 26670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:32.505967 26670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:32.506033 26670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:32.516021 26670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:32.517376 26670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:32.518687 26670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:32.531401 26678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:32.531474 26675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:32.531492 26676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:32.532577 26670 server_base.cc:1034] running on GCE node
I20260809 01:04:32.533439 26670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:32.534864 26670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:32.536100 26670 hybrid_clock.cc:648] HybridClock initialized: now 1786237472536040 us; error 55 us; skew 500 ppm
I20260809 01:04:32.536506 26670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:32 dist-test-slave-07c2 krb5kdc[26657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237472, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:32.539992 26670 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 01:04:32.541594 26670 webserver.cc:460] Webserver started at http://127.0.0.1:40815/ using document root <none> and password file <none>
I20260809 01:04:32.542313 26670 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:32.542455 26670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:32.542800 26670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:32.545909 26670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "41e14231019844c69fec50651a5fb92e"
format_stamp: "Formatted at 2026-08-09 01:04:32 on dist-test-slave-07c2"
server_key: "87c922257db8e926baf922ebb7beeb07"
server_key_iv: "8cea6e833cb75e38264f1ef49219f5e9"
server_key_version: "encryptionkey@0"
I20260809 01:04:32.546715 26670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "41e14231019844c69fec50651a5fb92e"
format_stamp: "Formatted at 2026-08-09 01:04:32 on dist-test-slave-07c2"
server_key: "87c922257db8e926baf922ebb7beeb07"
server_key_iv: "8cea6e833cb75e38264f1ef49219f5e9"
server_key_version: "encryptionkey@0"
I20260809 01:04:32.552392 26670 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 01:04:32.556056 26685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:32.557420 26670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 01:04:32.557623 26670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "41e14231019844c69fec50651a5fb92e"
format_stamp: "Formatted at 2026-08-09 01:04:32 on dist-test-slave-07c2"
server_key: "87c922257db8e926baf922ebb7beeb07"
server_key_iv: "8cea6e833cb75e38264f1ef49219f5e9"
server_key_version: "encryptionkey@0"
I20260809 01:04:32.557847 26670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:32.563819 26670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:32.568394 26670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:32.568742 26670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:32.588471 26670 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37095
I20260809 01:04:32.588519 26740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37095 every 8 connection(s)
I20260809 01:04:32.589893 26670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:32.590768 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 26670
I20260809 01:04:32.590994 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:32.591523 4426 external_mini_cluster.cc:1428] Setting key ade3080f5792c30c90d308c19d94c12d
I20260809 01:04:32.594449 26741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:32.600430 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:32.593405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51889 (local address 127.0.0.1:37095)
0809 01:04:32.594206 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:32.594220 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:32.594285 (+ 65us) server_negotiation.cc:407] Connection header received
0809 01:04:32.596675 (+ 2390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:32.596700 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:32.597099 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:32.597555 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:32.598527 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:32.598535 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:32.598550 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:32.598568 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:32.598666 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:32.598693 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:32.598916 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:32.599051 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 01:04:32.599230 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":327,"threads_started":1}
I20260809 01:04:32.604032 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:32.606871 26741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e: Bootstrap starting.
I20260809 01:04:32.610414 26741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:32.610971 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:32.608491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46619 (local address 127.0.0.1:37095)
0809 01:04:32.608698 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:32.608703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:32.609007 (+ 304us) server_negotiation.cc:407] Connection header received
0809 01:04:32.609133 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:32.609138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:32.609193 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:32.609305 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:32.610219 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:32.610228 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:32.610236 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:32.610254 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:32.610319 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:32.610342 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:32.610472 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:32.610605 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 01:04:32.610662 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 01:04:32.611696 26741 log.cc:824] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:32.614303 26741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e: No bootstrap required, opened a new log
I20260809 01:04:32.621253 26741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e14231019844c69fec50651a5fb92e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37095 } }
I20260809 01:04:32.621649 26741 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:32.621771 26741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e14231019844c69fec50651a5fb92e, State: Initialized, Role: FOLLOWER
I20260809 01:04:32.622430 26741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e14231019844c69fec50651a5fb92e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37095 } }
I20260809 01:04:32.622673 26741 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:32.622836 26741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:32.623025 26741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:32.625411 26741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e14231019844c69fec50651a5fb92e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37095 } }
I20260809 01:04:32.625910 26741 leader_election.cc:302] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41e14231019844c69fec50651a5fb92e; no voters:
I20260809 01:04:32.626514 26741 leader_election.cc:288] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:32.626668 26753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:32.627403 26753 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [term 1 LEADER]: Becoming Leader. State: Replica: 41e14231019844c69fec50651a5fb92e, State: Running, Role: LEADER
I20260809 01:04:32.627964 26753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e14231019844c69fec50651a5fb92e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37095 } }
I20260809 01:04:32.628649 26741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:32.631230 26754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41e14231019844c69fec50651a5fb92e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e14231019844c69fec50651a5fb92e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37095 } } }
I20260809 01:04:32.631824 26754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:32.631783 26755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41e14231019844c69fec50651a5fb92e. Latest consensus state: current_term: 1 leader_uuid: "41e14231019844c69fec50651a5fb92e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e14231019844c69fec50651a5fb92e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37095 } } }
I20260809 01:04:32.632498 26755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:32.635780 26759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:32.641076 26759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:32.649742 26759 catalog_manager.cc:1347] Generated new cluster ID: 2abbd011f751417fa5edafc571bb097a
I20260809 01:04:32.649859 26759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:32.664285 26759 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:32.664775 26759 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:32.679911 26759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41e14231019844c69fec50651a5fb92e: Generated new TSK 0
I20260809 01:04:32.680948 26759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:32.696777 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 26670
W20260809 01:04:32.698645 26736 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51889
W20260809 01:04:32.699484 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 26670: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 01:04:32.699636 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 26670
2026-08-09T01:04:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1983 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:32 dist-test-slave-07c2 krb5kdc[26776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:32 dist-test-slave-07c2 krb5kdc[26776](info): set up 2 sockets
Aug 09 01:04:32 dist-test-slave-07c2 krb5kdc[26776](info): commencing operation
krb5kdc: starting...
W20260809 01:04:33.859422 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.007s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:34 dist-test-slave-07c2 krb5kdc[26776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 01:04:34.174747 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33279
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:34.515552 26789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:34.515856 26789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:34.516017 26789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:34.523252 26789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:34.523447 26789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:34.523581 26789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:34.523689 26789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:34.523779 26789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:34.533921 26789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:34.535277 26789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:34.536659 26789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:34.548451 26794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:34.548583 26797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:34.548651 26795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:34.549619 26789 server_base.cc:1034] running on GCE node
I20260809 01:04:34.550401 26789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:34.551892 26789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:34.553148 26789 hybrid_clock.cc:648] HybridClock initialized: now 1786237474553108 us; error 35 us; skew 500 ppm
I20260809 01:04:34.553507 26789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:34 dist-test-slave-07c2 krb5kdc[26776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237474, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:34.556872 26789 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 01:04:34.558290 26789 webserver.cc:460] Webserver started at http://127.0.0.1:45623/ using document root <none> and password file <none>
I20260809 01:04:34.558933 26789 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:34.559031 26789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:34.559341 26789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:34.562218 26789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "adc7864a6c09438b85e3792db41c14b5"
format_stamp: "Formatted at 2026-08-09 01:04:34 on dist-test-slave-07c2"
server_key: "00a0495ff5012ee45f9805a13481d730"
server_key_iv: "08c991ff1dce2386aba004f9ebcb45c0"
server_key_version: "encryptionkey@0"
I20260809 01:04:34.562934 26789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "adc7864a6c09438b85e3792db41c14b5"
format_stamp: "Formatted at 2026-08-09 01:04:34 on dist-test-slave-07c2"
server_key: "00a0495ff5012ee45f9805a13481d730"
server_key_iv: "08c991ff1dce2386aba004f9ebcb45c0"
server_key_version: "encryptionkey@0"
I20260809 01:04:34.568207 26789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 01:04:34.571583 26805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:34.573024 26789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 01:04:34.573201 26789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "adc7864a6c09438b85e3792db41c14b5"
format_stamp: "Formatted at 2026-08-09 01:04:34 on dist-test-slave-07c2"
server_key: "00a0495ff5012ee45f9805a13481d730"
server_key_iv: "08c991ff1dce2386aba004f9ebcb45c0"
server_key_version: "encryptionkey@0"
I20260809 01:04:34.573369 26789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:34.591457 26789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:34.595888 26789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:34.596190 26789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:34.615046 26789 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33279
I20260809 01:04:34.615058 26861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33279 every 8 connection(s)
I20260809 01:04:34.616516 26789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:34.620150 26862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:34.625535 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 26789
I20260809 01:04:34.625739 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:34.626200 4426 external_mini_cluster.cc:1428] Setting key 2a8a6375df2b04ce75b22f8b1eabfd1a
I20260809 01:04:34.630753 26862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5: Bootstrap starting.
I20260809 01:04:34.634464 26862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:34.635751 26862 log.cc:824] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:34.638900 26862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5: No bootstrap required, opened a new log
Aug 09 01:04:34 dist-test-slave-07c2 krb5kdc[26776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 01:04:34.647082 26862 raft_consensus.cc:348] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc7864a6c09438b85e3792db41c14b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33279 } }
I20260809 01:04:34.647575 26862 raft_consensus.cc:374] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:34.647732 26862 raft_consensus.cc:729] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc7864a6c09438b85e3792db41c14b5, State: Initialized, Role: FOLLOWER
I20260809 01:04:34.648335 26862 consensus_queue.cc:260] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc7864a6c09438b85e3792db41c14b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33279 } }
I20260809 01:04:34.648643 26862 raft_consensus.cc:388] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:34.648826 26862 raft_consensus.cc:482] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:34.649027 26862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:34.651221 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:34.628037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:60005 (local address 127.0.0.1:33279)
0809 01:04:34.628890 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:34.628901 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:34.628945 (+ 44us) server_negotiation.cc:407] Connection header received
0809 01:04:34.630778 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:34.630799 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:34.631218 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:34.631636 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:34.633525 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:34.634869 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:34.636674 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:34.637471 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:34.637601 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:34.642528 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:34.642536 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:34.642553 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:34.642591 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:34.645981 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:34.646854 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:34.646861 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:34.646868 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:34.647041 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:34.647631 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:34.647635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:34.647637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:34.648396 (+ 759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:34.648797 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:34.649370 (+ 573us) server_negotiation.cc:299] Negotiation successful
0809 01:04:34.649736 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":422,"threads_started":1}
I20260809 01:04:34.652285 26862 raft_consensus.cc:504] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc7864a6c09438b85e3792db41c14b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33279 } }
I20260809 01:04:34.652823 26862 leader_election.cc:302] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adc7864a6c09438b85e3792db41c14b5; no voters:
I20260809 01:04:34.653519 26862 leader_election.cc:288] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:34.653671 26867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:34.654362 26867 raft_consensus.cc:686] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [term 1 LEADER]: Becoming Leader. State: Replica: adc7864a6c09438b85e3792db41c14b5, State: Running, Role: LEADER
I20260809 01:04:34.655175 26867 consensus_queue.cc:237] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc7864a6c09438b85e3792db41c14b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33279 } }
I20260809 01:04:34.655526 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:34.657085 26862 sys_catalog.cc:564] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:34.660003 26869 sys_catalog.cc:455] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adc7864a6c09438b85e3792db41c14b5. Latest consensus state: current_term: 1 leader_uuid: "adc7864a6c09438b85e3792db41c14b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc7864a6c09438b85e3792db41c14b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33279 } } }
I20260809 01:04:34.660679 26869 sys_catalog.cc:458] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:34.660760 26870 sys_catalog.cc:455] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adc7864a6c09438b85e3792db41c14b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc7864a6c09438b85e3792db41c14b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33279 } } }
I20260809 01:04:34.661211 26870 sys_catalog.cc:458] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:34.662226 26878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:34.675709 26878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:34.693071 26878 catalog_manager.cc:1347] Generated new cluster ID: 14d9dee66c33459f9e0cc72d7053714e
I20260809 01:04:34.693218 26878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:34.716136 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:34.686834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:57547 (local address 127.0.0.1:33279)
0809 01:04:34.687068 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:34.687078 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:34.691855 (+ 4777us) server_negotiation.cc:407] Connection header received
0809 01:04:34.692208 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:34.692218 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:34.692338 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:34.692430 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:34.695546 (+ 3116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:34.697523 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:34.700084 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:34.701098 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:34.701230 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:34.705933 (+ 4703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:34.705943 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:34.705950 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:34.706006 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:34.711081 (+ 5075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:34.712452 (+ 1371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:34.712460 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:34.712467 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:34.712711 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:34.713462 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:34.713470 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:34.713476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:34.714018 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:34.714386 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:34.715278 (+ 892us) server_negotiation.cc:299] Negotiation successful
0809 01:04:34.715563 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 01:04:34.729856 26878 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:34.731109 26878 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:34.736501 26878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adc7864a6c09438b85e3792db41c14b5: Generated new TSK 0
I20260809 01:04:34.737424 26878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:34.746109 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 26789
I20260809 01:04:34.747673 26857 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:34.839104 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 26789
W20260809 01:04:34.859483 26774 connection.cc:535] client connection to 127.0.0.1:33279 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33279): Connection reset by peer (error 104)
2026-08-09T01:04:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2140 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:34 dist-test-slave-07c2 krb5kdc[26896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:34 dist-test-slave-07c2 krb5kdc[26896](info): set up 2 sockets
Aug 09 01:04:34 dist-test-slave-07c2 krb5kdc[26896](info): commencing operation
krb5kdc: starting...
W20260809 01:04:36.003623 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.006s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:36 dist-test-slave-07c2 krb5kdc[26896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 01:04:36.334721 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35179
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:36.688030 26909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:36.688375 26909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:36.688586 26909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:36.695806 26909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:36.695964 26909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:36.696070 26909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:36.696188 26909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:36.696280 26909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:36.706027 26909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:36.707355 26909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:36.708720 26909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:36.720495 26914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:36.720788 26917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:36.720806 26915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:36.721594 26909 server_base.cc:1034] running on GCE node
I20260809 01:04:36.722389 26909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:36.723814 26909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:36.725013 26909 hybrid_clock.cc:648] HybridClock initialized: now 1786237476724981 us; error 30 us; skew 500 ppm
I20260809 01:04:36.725370 26909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:36 dist-test-slave-07c2 krb5kdc[26896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237476, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:36.734143 26909 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 01:04:36.735843 26909 webserver.cc:460] Webserver started at http://127.0.0.1:43417/ using document root <none> and password file <none>
I20260809 01:04:36.736575 26909 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:36.736734 26909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:36.737056 26909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:36.740525 26909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "a68bd6665cea47ba8ce4bdffbc687fd3"
format_stamp: "Formatted at 2026-08-09 01:04:36 on dist-test-slave-07c2"
server_key: "5bd6b99d60e2e3989ac8c4b55a35c7fe"
server_key_iv: "eae4716a722ab56dc04df49161ffa3e4"
server_key_version: "encryptionkey@0"
I20260809 01:04:36.741354 26909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "a68bd6665cea47ba8ce4bdffbc687fd3"
format_stamp: "Formatted at 2026-08-09 01:04:36 on dist-test-slave-07c2"
server_key: "5bd6b99d60e2e3989ac8c4b55a35c7fe"
server_key_iv: "eae4716a722ab56dc04df49161ffa3e4"
server_key_version: "encryptionkey@0"
I20260809 01:04:36.746575 26909 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 01:04:36.750154 26925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:36.751577 26909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:04:36.751770 26909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "a68bd6665cea47ba8ce4bdffbc687fd3"
format_stamp: "Formatted at 2026-08-09 01:04:36 on dist-test-slave-07c2"
server_key: "5bd6b99d60e2e3989ac8c4b55a35c7fe"
server_key_iv: "eae4716a722ab56dc04df49161ffa3e4"
server_key_version: "encryptionkey@0"
I20260809 01:04:36.751955 26909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:36.770464 26909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:36.774832 26909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:36.775121 26909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:36.793467 26909 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35179
I20260809 01:04:36.793516 26980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35179 every 8 connection(s)
I20260809 01:04:36.794865 26909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:36.797106 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 26909
I20260809 01:04:36.797312 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:36.797890 4426 external_mini_cluster.cc:1428] Setting key 71fc93b74ac8c9b2b0e2ee9f701fedd4
I20260809 01:04:36.799470 26981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:36.813172 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:36.799819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:34095 (local address 127.0.0.1:35179)
0809 01:04:36.800579 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:36.800594 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:36.800643 (+ 49us) server_negotiation.cc:407] Connection header received
0809 01:04:36.802693 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:36.802728 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:36.803248 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:36.803709 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:36.804241 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:36.806325 (+ 2084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:36.807773 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:36.808835 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:36.809010 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:36.810246 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:36.810257 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:36.810275 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:36.811039 (+ 764us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:36.811133 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:36.811171 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:36.811360 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:36.811675 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 01:04:36.811865 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":299,"threads_started":1}
I20260809 01:04:36.814689 26981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3: Bootstrap starting.
I20260809 01:04:36.817374 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:36.818152 26981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:36.819144 26981 log.cc:824] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:36.822592 26981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3: No bootstrap required, opened a new log
I20260809 01:04:36.828593 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:36.822084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:52887 (local address 127.0.0.1:35179)
0809 01:04:36.822279 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:36.822287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:36.822780 (+ 493us) server_negotiation.cc:407] Connection header received
0809 01:04:36.822979 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:36.822987 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:36.823056 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:36.823187 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:36.823737 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:36.824544 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:36.825917 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:36.826368 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:36.826535 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:36.827820 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:36.827825 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:36.827828 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:36.827849 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:36.827889 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:36.827904 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:36.828043 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:36.828251 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 01:04:36.828309 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 01:04:36.830840 26981 raft_consensus.cc:348] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68bd6665cea47ba8ce4bdffbc687fd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35179 } }
I20260809 01:04:36.831254 26981 raft_consensus.cc:374] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:36.831404 26981 raft_consensus.cc:729] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a68bd6665cea47ba8ce4bdffbc687fd3, State: Initialized, Role: FOLLOWER
I20260809 01:04:36.831913 26981 consensus_queue.cc:260] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68bd6665cea47ba8ce4bdffbc687fd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35179 } }
I20260809 01:04:36.832118 26981 raft_consensus.cc:388] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:36.832221 26981 raft_consensus.cc:482] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:36.832376 26981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:36.834836 26981 raft_consensus.cc:504] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68bd6665cea47ba8ce4bdffbc687fd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35179 } }
I20260809 01:04:36.835279 26981 leader_election.cc:302] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a68bd6665cea47ba8ce4bdffbc687fd3; no voters:
I20260809 01:04:36.835884 26981 leader_election.cc:288] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:36.836028 26993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:36.837071 26993 raft_consensus.cc:686] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [term 1 LEADER]: Becoming Leader. State: Replica: a68bd6665cea47ba8ce4bdffbc687fd3, State: Running, Role: LEADER
I20260809 01:04:36.837643 26993 consensus_queue.cc:237] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68bd6665cea47ba8ce4bdffbc687fd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35179 } }
I20260809 01:04:36.838157 26981 sys_catalog.cc:564] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:36.840287 26995 sys_catalog.cc:455] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a68bd6665cea47ba8ce4bdffbc687fd3. Latest consensus state: current_term: 1 leader_uuid: "a68bd6665cea47ba8ce4bdffbc687fd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68bd6665cea47ba8ce4bdffbc687fd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35179 } } }
I20260809 01:04:36.840689 26995 sys_catalog.cc:458] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:36.840495 26994 sys_catalog.cc:455] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a68bd6665cea47ba8ce4bdffbc687fd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a68bd6665cea47ba8ce4bdffbc687fd3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35179 } } }
I20260809 01:04:36.840996 26994 sys_catalog.cc:458] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:36.842451 27000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:36.847656 27000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:36.857374 27000 catalog_manager.cc:1347] Generated new cluster ID: e51a6ba987ac4436b7f20bcd244467c9
I20260809 01:04:36.857492 27000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:36.872153 27000 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:36.873831 27000 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:36.882419 27000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a68bd6665cea47ba8ce4bdffbc687fd3: Generated new TSK 0
I20260809 01:04:36.883229 27000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:36.888388 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 26909
W20260809 01:04:36.889528 26976 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.85:34095
W20260809 01:04:36.890286 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 26909: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 01:04:36.890444 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 26909
W20260809 01:04:36.912107 26894 connection.cc:535] client connection to 127.0.0.1:35179 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35179): Connection reset by peer (error 104)
2026-08-09T01:04:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2052 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:36 dist-test-slave-07c2 krb5kdc[27017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:37 dist-test-slave-07c2 krb5kdc[27017](info): set up 2 sockets
Aug 09 01:04:37 dist-test-slave-07c2 krb5kdc[27017](info): commencing operation
krb5kdc: starting...
W20260809 01:04:38.046399 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:38 dist-test-slave-07c2 krb5kdc[27017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 01:04:38.339025 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35839
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:38.699174 27031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:38.699644 27031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:38.699761 27031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:38.708221 27031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:38.708427 27031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:38.708619 27031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:38.708760 27031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:38.708850 27031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:38.720134 27031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:38.721534 27031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:38.722891 27031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:38.735443 27039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:38.735579 27036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:38.735538 27037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:38.736634 27031 server_base.cc:1034] running on GCE node
I20260809 01:04:38.737787 27031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:38.739336 27031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:38.740576 27031 hybrid_clock.cc:648] HybridClock initialized: now 1786237478740529 us; error 57 us; skew 500 ppm
I20260809 01:04:38.740953 27031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:38 dist-test-slave-07c2 krb5kdc[27017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237478, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:38.744447 27031 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 01:04:38.746138 27031 webserver.cc:460] Webserver started at http://127.0.0.1:36249/ using document root <none> and password file <none>
I20260809 01:04:38.746892 27031 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:38.747057 27031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:38.747483 27031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:38.750705 27031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "00f475d898a64ed7b12a1f53fd9530b9"
format_stamp: "Formatted at 2026-08-09 01:04:38 on dist-test-slave-07c2"
server_key: "e12bf337f95d4c96c964694ad94a480a"
server_key_iv: "1780fa504b39c3ecf21f1b804f70505b"
server_key_version: "encryptionkey@0"
I20260809 01:04:38.751626 27031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "00f475d898a64ed7b12a1f53fd9530b9"
format_stamp: "Formatted at 2026-08-09 01:04:38 on dist-test-slave-07c2"
server_key: "e12bf337f95d4c96c964694ad94a480a"
server_key_iv: "1780fa504b39c3ecf21f1b804f70505b"
server_key_version: "encryptionkey@0"
I20260809 01:04:38.757215 27031 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 01:04:38.761094 27047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:38.762465 27031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 01:04:38.762691 27031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "00f475d898a64ed7b12a1f53fd9530b9"
format_stamp: "Formatted at 2026-08-09 01:04:38 on dist-test-slave-07c2"
server_key: "e12bf337f95d4c96c964694ad94a480a"
server_key_iv: "1780fa504b39c3ecf21f1b804f70505b"
server_key_version: "encryptionkey@0"
I20260809 01:04:38.762928 27031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:38.769114 27031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:38.773826 27031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:38.774164 27031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:38.794111 27031 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35839
I20260809 01:04:38.794193 27103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35839 every 8 connection(s)
I20260809 01:04:38.795734 27031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:38.799685 27105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:38.801898 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 27031
I20260809 01:04:38.802095 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:38.802577 4426 external_mini_cluster.cc:1428] Setting key cb01d91dd37766bce34e4360f3606220
I20260809 01:04:38.810966 27108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:38.804480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:43193 (local address 127.0.0.1:35839)
0809 01:04:38.805349 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:38.805360 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:38.805410 (+ 50us) server_negotiation.cc:407] Connection header received
0809 01:04:38.807078 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:38.807101 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:38.807510 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:38.807844 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:38.809025 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:38.809061 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:38.809085 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:38.809107 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:38.809194 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:38.809228 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:38.809481 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:38.809640 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 01:04:38.809907 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":413,"threads_started":1}
I20260809 01:04:38.812533 27105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9: Bootstrap starting.
I20260809 01:04:38.814905 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:38.816087 27105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:38.817145 27105 log.cc:824] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:38.820359 27105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9: No bootstrap required, opened a new log
I20260809 01:04:38.822738 27108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:38.819782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:41019 (local address 127.0.0.1:35839)
0809 01:04:38.819984 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:38.819991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:38.820596 (+ 605us) server_negotiation.cc:407] Connection header received
0809 01:04:38.820796 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:38.820800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:38.820851 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:38.820932 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:38.822003 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:38.822007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:38.822010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:38.822026 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:38.822064 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:38.822079 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:38.822173 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:38.822334 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 01:04:38.822389 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 01:04:38.829077 27105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00f475d898a64ed7b12a1f53fd9530b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35839 } }
I20260809 01:04:38.829526 27105 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:38.829658 27105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00f475d898a64ed7b12a1f53fd9530b9, State: Initialized, Role: FOLLOWER
I20260809 01:04:38.830333 27105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00f475d898a64ed7b12a1f53fd9530b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35839 } }
I20260809 01:04:38.830597 27105 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:38.830752 27105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:38.830940 27105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:38.833284 27105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00f475d898a64ed7b12a1f53fd9530b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35839 } }
I20260809 01:04:38.833827 27105 leader_election.cc:302] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00f475d898a64ed7b12a1f53fd9530b9; no voters:
I20260809 01:04:38.834534 27105 leader_election.cc:288] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:38.834654 27115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:38.835794 27115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [term 1 LEADER]: Becoming Leader. State: Replica: 00f475d898a64ed7b12a1f53fd9530b9, State: Running, Role: LEADER
I20260809 01:04:38.836383 27115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00f475d898a64ed7b12a1f53fd9530b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35839 } }
I20260809 01:04:38.837110 27105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:38.839473 27116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00f475d898a64ed7b12a1f53fd9530b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00f475d898a64ed7b12a1f53fd9530b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35839 } } }
I20260809 01:04:38.839887 27116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:38.844791 27117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00f475d898a64ed7b12a1f53fd9530b9. Latest consensus state: current_term: 1 leader_uuid: "00f475d898a64ed7b12a1f53fd9530b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00f475d898a64ed7b12a1f53fd9530b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35839 } } }
I20260809 01:04:38.845028 27117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:38.845696 27122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:38.851137 27122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:38.861357 27122 catalog_manager.cc:1347] Generated new cluster ID: 027453965f6a4f40bb825f279f9cbcf1
I20260809 01:04:38.861487 27122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:38.882371 27122 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:38.882946 27122 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:38.891032 27122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00f475d898a64ed7b12a1f53fd9530b9: Generated new TSK 0
I20260809 01:04:38.892094 27122 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:38.899518 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27031
W20260809 01:04:38.900655 27098 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.85:43193
W20260809 01:04:38.901398 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27031: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 01:04:38.901557 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 27031
2026-08-09T01:04:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2006 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:39 dist-test-slave-07c2 krb5kdc[27137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:39 dist-test-slave-07c2 krb5kdc[27137](info): set up 2 sockets
Aug 09 01:04:39 dist-test-slave-07c2 krb5kdc[27137](info): commencing operation
krb5kdc: starting...
W20260809 01:04:40.055687 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:40 dist-test-slave-07c2 krb5kdc[27137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 01:04:40.382856 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:40.754309 27152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:40.754712 27152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:40.754901 27152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:40.762387 27152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:40.762567 27152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:40.762707 27152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:40.762822 27152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:40.762909 27152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:40.773006 27152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32807
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:40.774423 27152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:40.775831 27152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:40.787940 27160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:40.788199 27158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:40.788352 27157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:40.788913 27152 server_base.cc:1034] running on GCE node
I20260809 01:04:40.789788 27152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:40.791379 27152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:40.792639 27152 hybrid_clock.cc:648] HybridClock initialized: now 1786237480792597 us; error 48 us; skew 500 ppm
I20260809 01:04:40.793023 27152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:40 dist-test-slave-07c2 krb5kdc[27137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237480, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:40.796649 27152 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 01:04:40.798427 27152 webserver.cc:460] Webserver started at http://127.0.0.1:35461/ using document root <none> and password file <none>
I20260809 01:04:40.799201 27152 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:40.799387 27152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:40.799723 27152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:40.802910 27152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "5202a55078dc4c64aa3d61751b0ea2ee"
format_stamp: "Formatted at 2026-08-09 01:04:40 on dist-test-slave-07c2"
server_key: "634f7fc22f44f17e18a3d3715cd55e2d"
server_key_iv: "3b7c0bd4c36cead557f7dfee8cc6314b"
server_key_version: "encryptionkey@0"
I20260809 01:04:40.803791 27152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "5202a55078dc4c64aa3d61751b0ea2ee"
format_stamp: "Formatted at 2026-08-09 01:04:40 on dist-test-slave-07c2"
server_key: "634f7fc22f44f17e18a3d3715cd55e2d"
server_key_iv: "3b7c0bd4c36cead557f7dfee8cc6314b"
server_key_version: "encryptionkey@0"
I20260809 01:04:40.809290 27152 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 01:04:40.813135 27167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:40.814565 27152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 01:04:40.814800 27152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "5202a55078dc4c64aa3d61751b0ea2ee"
format_stamp: "Formatted at 2026-08-09 01:04:40 on dist-test-slave-07c2"
server_key: "634f7fc22f44f17e18a3d3715cd55e2d"
server_key_iv: "3b7c0bd4c36cead557f7dfee8cc6314b"
server_key_version: "encryptionkey@0"
I20260809 01:04:40.815042 27152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:40.867244 27152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:40.871979 27152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:40.872303 27152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:40.892030 27152 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32807
I20260809 01:04:40.892087 27223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32807 every 8 connection(s)
I20260809 01:04:40.893482 27152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:40.896750 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 27152
I20260809 01:04:40.896975 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:40.897423 27224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:40.897447 4426 external_mini_cluster.cc:1428] Setting key 496555e8056edb543289f95b76ff7407
I20260809 01:04:40.912245 27224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee: Bootstrap starting.
Aug 09 01:04:40 dist-test-slave-07c2 krb5kdc[27137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 01:04:40.916306 27224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:40.917763 27224 log.cc:824] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:40.921495 27224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee: No bootstrap required, opened a new log
I20260809 01:04:40.922925 27227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:40.899262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:58519 (local address 127.0.0.1:32807)
0809 01:04:40.900131 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:40.900142 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:40.900186 (+ 44us) server_negotiation.cc:407] Connection header received
0809 01:04:40.902057 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:40.902096 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:40.902507 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:40.902840 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:40.904773 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:40.906037 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:40.907898 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:40.908626 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:40.908780 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:40.913801 (+ 5021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:40.913818 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:40.913839 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:40.913912 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:40.918003 (+ 4091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:40.919039 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:40.919045 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:40.919051 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:40.919243 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:40.919750 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:40.919755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:40.919758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:40.920484 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:40.920805 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:40.921235 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 01:04:40.921569 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":379,"threads_started":1}
I20260809 01:04:40.926918 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:40.930295 27224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5202a55078dc4c64aa3d61751b0ea2ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32807 } }
I20260809 01:04:40.930691 27224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:40.930809 27224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5202a55078dc4c64aa3d61751b0ea2ee, State: Initialized, Role: FOLLOWER
I20260809 01:04:40.931532 27224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5202a55078dc4c64aa3d61751b0ea2ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32807 } }
I20260809 01:04:40.931912 27224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:40.932070 27224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:40.932255 27224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:40.934989 27224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5202a55078dc4c64aa3d61751b0ea2ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32807 } }
I20260809 01:04:40.935542 27224 leader_election.cc:302] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5202a55078dc4c64aa3d61751b0ea2ee; no voters:
I20260809 01:04:40.936178 27224 leader_election.cc:288] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:40.936319 27234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:40.936997 27234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [term 1 LEADER]: Becoming Leader. State: Replica: 5202a55078dc4c64aa3d61751b0ea2ee, State: Running, Role: LEADER
I20260809 01:04:40.937670 27234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5202a55078dc4c64aa3d61751b0ea2ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32807 } }
I20260809 01:04:40.939965 27224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:40.940517 27235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5202a55078dc4c64aa3d61751b0ea2ee. Latest consensus state: current_term: 1 leader_uuid: "5202a55078dc4c64aa3d61751b0ea2ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5202a55078dc4c64aa3d61751b0ea2ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32807 } } }
I20260809 01:04:40.940860 27235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:40.941689 27236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5202a55078dc4c64aa3d61751b0ea2ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5202a55078dc4c64aa3d61751b0ea2ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32807 } } }
I20260809 01:04:40.941928 27238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:40.942082 27236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:40.951113 27227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:40.931622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:55309 (local address 127.0.0.1:32807)
0809 01:04:40.931824 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:40.931832 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:40.932312 (+ 480us) server_negotiation.cc:407] Connection header received
0809 01:04:40.932494 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:40.932500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:40.932587 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:40.932697 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:40.934030 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:40.935135 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:40.936946 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:40.937626 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:40.937776 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:40.941389 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:40.941398 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:40.941406 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:40.941452 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:40.945424 (+ 3972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:40.946722 (+ 1298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:40.946730 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:40.946736 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:40.946918 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:40.947695 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:40.947703 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:40.947708 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:40.948276 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:40.950062 (+ 1786us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:40.950306 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 01:04:40.950557 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 01:04:40.955188 27238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:40.964754 27238 catalog_manager.cc:1347] Generated new cluster ID: 0dcb905308e847d9b126e17b0b07b02c
I20260809 01:04:40.964964 27238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:40.997367 27238 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:40.998711 27238 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:41.014503 27238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5202a55078dc4c64aa3d61751b0ea2ee: Generated new TSK 0
I20260809 01:04:41.015561 27238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:41.033305 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27152
I20260809 01:04:41.034513 27218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:41.130760 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 27152
W20260809 01:04:41.151108 27134 connection.cc:535] client connection to 127.0.0.1:32807 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32807): Connection reset by peer (error 104)
2026-08-09T01:04:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2232 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:41 dist-test-slave-07c2 krb5kdc[27257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:41 dist-test-slave-07c2 krb5kdc[27257](info): set up 2 sockets
Aug 09 01:04:41 dist-test-slave-07c2 krb5kdc[27257](info): commencing operation
krb5kdc: starting...
W20260809 01:04:42.296080 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:42 dist-test-slave-07c2 krb5kdc[27257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 01:04:42.626924 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44997
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:42.970898 27271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:42.971174 27271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:42.971390 27271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:42.978564 27271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:42.978708 27271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:42.978821 27271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:42.978930 27271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:42.979014 27271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:42.989293 27271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44997
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:42.990530 27271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:42.991876 27271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:43.003540 27279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:43.003620 27277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:43.003577 27281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:43.004531 27271 server_base.cc:1034] running on GCE node
I20260809 01:04:43.005316 27271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:43.006714 27271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:43.007938 27271 hybrid_clock.cc:648] HybridClock initialized: now 1786237483007879 us; error 47 us; skew 500 ppm
I20260809 01:04:43.008277 27271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:43 dist-test-slave-07c2 krb5kdc[27257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237483, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:43.015997 27271 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 01:04:43.017544 27271 webserver.cc:460] Webserver started at http://127.0.0.1:33217/ using document root <none> and password file <none>
I20260809 01:04:43.018226 27271 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:43.018388 27271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:43.018721 27271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:43.021745 27271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "68b274af8029497a8561b6aee5318755"
format_stamp: "Formatted at 2026-08-09 01:04:43 on dist-test-slave-07c2"
server_key: "515480a2e3d37ce37b8348f1b96dad51"
server_key_iv: "2edb96f4c96e73a11de88af5c3e1cca1"
server_key_version: "encryptionkey@0"
I20260809 01:04:43.022516 27271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "68b274af8029497a8561b6aee5318755"
format_stamp: "Formatted at 2026-08-09 01:04:43 on dist-test-slave-07c2"
server_key: "515480a2e3d37ce37b8348f1b96dad51"
server_key_iv: "2edb96f4c96e73a11de88af5c3e1cca1"
server_key_version: "encryptionkey@0"
I20260809 01:04:43.027700 27271 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260809 01:04:43.031136 27288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:43.032449 27271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:04:43.032660 27271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "68b274af8029497a8561b6aee5318755"
format_stamp: "Formatted at 2026-08-09 01:04:43 on dist-test-slave-07c2"
server_key: "515480a2e3d37ce37b8348f1b96dad51"
server_key_iv: "2edb96f4c96e73a11de88af5c3e1cca1"
server_key_version: "encryptionkey@0"
I20260809 01:04:43.032867 27271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:43.052736 27271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:43.057147 27271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:43.057461 27271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:43.077792 27271 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44997
I20260809 01:04:43.077826 27342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44997 every 8 connection(s)
I20260809 01:04:43.079416 27271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:43.083580 27343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:43.084430 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 27271
I20260809 01:04:43.084630 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:43.085131 4426 external_mini_cluster.cc:1428] Setting key 7b7eaa88c9f956c951a962db9347877b
I20260809 01:04:43.099272 27343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755: Bootstrap starting.
I20260809 01:04:43.101891 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:43.087151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:46619 (local address 127.0.0.1:44997)
0809 01:04:43.088084 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:43.088100 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:43.088153 (+ 53us) server_negotiation.cc:407] Connection header received
0809 01:04:43.090912 (+ 2759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:43.090945 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:43.091550 (+ 605us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:43.092034 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:43.092799 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:43.094798 (+ 1999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:43.096378 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:43.097492 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:43.097679 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:43.098952 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:43.098962 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:43.098983 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:43.099754 (+ 771us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:43.099860 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:43.099901 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:43.100057 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:43.100329 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 01:04:43.100501 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":356,"threads_started":1}
I20260809 01:04:43.103156 27343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:43.104290 27343 log.cc:824] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:43.106036 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:43.107836 27343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755: No bootstrap required, opened a new log
I20260809 01:04:43.118481 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:43.111536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:39297 (local address 127.0.0.1:44997)
0809 01:04:43.111728 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:43.111735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:43.112106 (+ 371us) server_negotiation.cc:407] Connection header received
0809 01:04:43.112281 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:43.112286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:43.112370 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:43.112488 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:43.113010 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:43.114224 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:43.115688 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:43.116420 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:43.116565 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:43.117688 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:43.117695 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:43.117699 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:43.117721 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:43.117765 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:43.117788 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:43.117905 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:43.118080 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 01:04:43.118145 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 01:04:43.118139 27343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b274af8029497a8561b6aee5318755" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44997 } }
I20260809 01:04:43.118664 27343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:43.118803 27343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b274af8029497a8561b6aee5318755, State: Initialized, Role: FOLLOWER
I20260809 01:04:43.119444 27343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b274af8029497a8561b6aee5318755" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44997 } }
I20260809 01:04:43.119725 27343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:43.119897 27343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:43.120096 27343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:43.122303 27343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b274af8029497a8561b6aee5318755" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44997 } }
I20260809 01:04:43.122820 27343 leader_election.cc:302] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68b274af8029497a8561b6aee5318755; no voters:
I20260809 01:04:43.123518 27343 leader_election.cc:288] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:43.123601 27354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:43.124402 27354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [term 1 LEADER]: Becoming Leader. State: Replica: 68b274af8029497a8561b6aee5318755, State: Running, Role: LEADER
I20260809 01:04:43.125087 27354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b274af8029497a8561b6aee5318755" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44997 } }
I20260809 01:04:43.125936 27343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:43.128640 27355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68b274af8029497a8561b6aee5318755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b274af8029497a8561b6aee5318755" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44997 } } }
I20260809 01:04:43.129140 27355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:43.129616 27356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68b274af8029497a8561b6aee5318755. Latest consensus state: current_term: 1 leader_uuid: "68b274af8029497a8561b6aee5318755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b274af8029497a8561b6aee5318755" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44997 } } }
I20260809 01:04:43.129976 27356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:43.138000 27363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:43.143255 27363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:43.152297 27363 catalog_manager.cc:1347] Generated new cluster ID: 226e616f95e543738d5c12bb334b99bc
I20260809 01:04:43.152436 27363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:43.170197 27363 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:43.171519 27363 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:43.178503 27363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68b274af8029497a8561b6aee5318755: Generated new TSK 0
I20260809 01:04:43.179234 27363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:43.188896 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27271
W20260809 01:04:43.190081 27338 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.85:46619
W20260809 01:04:43.190848 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27271: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 01:04:43.191002 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 27271
W20260809 01:04:43.210541 27254 connection.cc:535] client connection to 127.0.0.1:44997 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44997): Connection reset by peer (error 104)
2026-08-09T01:04:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2059 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:43 dist-test-slave-07c2 krb5kdc[27375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:43 dist-test-slave-07c2 krb5kdc[27375](info): set up 2 sockets
Aug 09 01:04:43 dist-test-slave-07c2 krb5kdc[27375](info): commencing operation
krb5kdc: starting...
W20260809 01:04:44.348109 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.002s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:44 dist-test-slave-07c2 krb5kdc[27375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 01:04:44.644855 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39329
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:44.979255 27388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:44.979614 27388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:44.979804 27388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:44.986871 27388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:44.987015 27388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:44.987126 27388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:44.987239 27388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:44.987345 27388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:44.996918 27388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39329
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:44.998158 27388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:44.999454 27388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:45.010819 27394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:45.010852 27397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:45.010892 27395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:45.011965 27388 server_base.cc:1034] running on GCE node
I20260809 01:04:45.012669 27388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:45.014043 27388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:45.015255 27388 hybrid_clock.cc:648] HybridClock initialized: now 1786237485015216 us; error 28 us; skew 500 ppm
I20260809 01:04:45.015611 27388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:45 dist-test-slave-07c2 krb5kdc[27375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237485, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:45.018565 27388 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 01:04:45.019986 27388 webserver.cc:460] Webserver started at http://127.0.0.1:38785/ using document root <none> and password file <none>
I20260809 01:04:45.020675 27388 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:45.020778 27388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:45.021019 27388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:45.023945 27388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "05775713c1ac4b0f914364f9ebfa1772"
format_stamp: "Formatted at 2026-08-09 01:04:45 on dist-test-slave-07c2"
server_key: "1389b66e28cc120c3277095afbdf28df"
server_key_iv: "6ead24208e3709bcbc551a71081e034f"
server_key_version: "encryptionkey@0"
I20260809 01:04:45.024669 27388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "05775713c1ac4b0f914364f9ebfa1772"
format_stamp: "Formatted at 2026-08-09 01:04:45 on dist-test-slave-07c2"
server_key: "1389b66e28cc120c3277095afbdf28df"
server_key_iv: "6ead24208e3709bcbc551a71081e034f"
server_key_version: "encryptionkey@0"
I20260809 01:04:45.029623 27388 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 01:04:45.032982 27405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:45.034235 27388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 01:04:45.034420 27388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "05775713c1ac4b0f914364f9ebfa1772"
format_stamp: "Formatted at 2026-08-09 01:04:45 on dist-test-slave-07c2"
server_key: "1389b66e28cc120c3277095afbdf28df"
server_key_iv: "6ead24208e3709bcbc551a71081e034f"
server_key_version: "encryptionkey@0"
I20260809 01:04:45.034608 27388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:45.040135 27388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:45.044121 27388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:45.044423 27388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:45.063352 27388 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39329
I20260809 01:04:45.063369 27459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39329 every 8 connection(s)
I20260809 01:04:45.064777 27388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:45.068444 27460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:45.071383 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 27388
I20260809 01:04:45.071566 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:45.072106 4426 external_mini_cluster.cc:1428] Setting key 39a39c4402e63826185d2370d1f502f5
I20260809 01:04:45.081636 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:45.074292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:60003 (local address 127.0.0.1:39329)
0809 01:04:45.075241 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:45.075256 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:45.075677 (+ 421us) server_negotiation.cc:407] Connection header received
0809 01:04:45.077808 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:45.077840 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:45.078284 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:45.078711 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:45.079735 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:45.079743 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:45.079759 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:45.079779 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:45.079854 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:45.079877 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:45.080099 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:45.080231 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 01:04:45.080431 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":410,"threads_started":1}
I20260809 01:04:45.082401 27460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772: Bootstrap starting.
I20260809 01:04:45.085301 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:45.085788 27460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:45.087000 27460 log.cc:824] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:45.090346 27460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772: No bootstrap required, opened a new log
I20260809 01:04:45.093323 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:45.090668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.85:58313 (local address 127.0.0.1:39329)
0809 01:04:45.090842 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:45.090853 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:45.091358 (+ 505us) server_negotiation.cc:407] Connection header received
0809 01:04:45.091544 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:45.091548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:45.091594 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:45.091673 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:45.092630 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:45.092634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:45.092637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:45.092654 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:45.092694 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:45.092709 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:45.092798 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:45.092961 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 01:04:45.093010 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 01:04:45.099036 27460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05775713c1ac4b0f914364f9ebfa1772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39329 } }
I20260809 01:04:45.099471 27460 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:45.099589 27460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05775713c1ac4b0f914364f9ebfa1772, State: Initialized, Role: FOLLOWER
I20260809 01:04:45.100214 27460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05775713c1ac4b0f914364f9ebfa1772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39329 } }
I20260809 01:04:45.100544 27460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:45.100721 27460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:45.100915 27460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:45.103101 27460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05775713c1ac4b0f914364f9ebfa1772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39329 } }
I20260809 01:04:45.103618 27460 leader_election.cc:302] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05775713c1ac4b0f914364f9ebfa1772; no voters:
I20260809 01:04:45.104220 27460 leader_election.cc:288] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:45.104384 27470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:45.105239 27470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [term 1 LEADER]: Becoming Leader. State: Replica: 05775713c1ac4b0f914364f9ebfa1772, State: Running, Role: LEADER
I20260809 01:04:45.105767 27470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05775713c1ac4b0f914364f9ebfa1772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39329 } }
I20260809 01:04:45.106413 27460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:45.109203 27472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05775713c1ac4b0f914364f9ebfa1772. Latest consensus state: current_term: 1 leader_uuid: "05775713c1ac4b0f914364f9ebfa1772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05775713c1ac4b0f914364f9ebfa1772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39329 } } }
I20260809 01:04:45.109689 27472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:45.109411 27471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05775713c1ac4b0f914364f9ebfa1772" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05775713c1ac4b0f914364f9ebfa1772" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39329 } } }
I20260809 01:04:45.109930 27471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:45.114575 27476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:45.120220 27476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:45.128783 27476 catalog_manager.cc:1347] Generated new cluster ID: cf0e60d8d65b40258d2ae1f682a46ea9
I20260809 01:04:45.128897 27476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:45.164211 27476 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:45.164692 27476 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:45.178277 27476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05775713c1ac4b0f914364f9ebfa1772: Generated new TSK 0
I20260809 01:04:45.179318 27476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:45.190089 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27388
W20260809 01:04:45.191155 27455 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.85:60003
W20260809 01:04:45.191959 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27388: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 01:04:45.192122 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 27388
2026-08-09T01:04:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2001 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:45 dist-test-slave-07c2 krb5kdc[27494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:45 dist-test-slave-07c2 krb5kdc[27494](info): set up 2 sockets
Aug 09 01:04:45 dist-test-slave-07c2 krb5kdc[27494](info): commencing operation
krb5kdc: starting...
W20260809 01:04:46.343575 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:46 dist-test-slave-07c2 krb5kdc[27494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:04:46.670852 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:44035
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:44035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:47.015187 27507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:47.015502 27507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:47.015606 27507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:47.024068 27507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:47.024200 27507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:47.024248 27507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:47.024319 27507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:47.024363 27507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:47.035509 27507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:44035
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:44035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:47.036816 27507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:47.038100 27507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:47.050020 27512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:47.050093 27513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:47.050124 27515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:47.051333 27507 server_base.cc:1034] running on GCE node
I20260809 01:04:47.052145 27507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:47.053512 27507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:47.054726 27507 hybrid_clock.cc:648] HybridClock initialized: now 1786237487054680 us; error 38 us; skew 500 ppm
I20260809 01:04:47.055085 27507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:47 dist-test-slave-07c2 krb5kdc[27494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237487, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:47.058475 27507 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:04:47.060086 27507 webserver.cc:460] Webserver started at http://127.4.82.190:42543/ using document root <none> and password file <none>
I20260809 01:04:47.060779 27507 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:47.060930 27507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:47.061244 27507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:47.064468 27507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "c86f81a9955d4fb6bb9c951ae1386a0e"
format_stamp: "Formatted at 2026-08-09 01:04:47 on dist-test-slave-07c2"
server_key: "e2bb34ef18e4c22809767b7db43e6d6e"
server_key_iv: "4894e2e0b0952c9d7c120433d1a5c215"
server_key_version: "encryptionkey@0"
I20260809 01:04:47.065450 27507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "c86f81a9955d4fb6bb9c951ae1386a0e"
format_stamp: "Formatted at 2026-08-09 01:04:47 on dist-test-slave-07c2"
server_key: "e2bb34ef18e4c22809767b7db43e6d6e"
server_key_iv: "4894e2e0b0952c9d7c120433d1a5c215"
server_key_version: "encryptionkey@0"
I20260809 01:04:47.070978 27507 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 01:04:47.074563 27522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:47.075920 27507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:04:47.076115 27507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "c86f81a9955d4fb6bb9c951ae1386a0e"
format_stamp: "Formatted at 2026-08-09 01:04:47 on dist-test-slave-07c2"
server_key: "e2bb34ef18e4c22809767b7db43e6d6e"
server_key_iv: "4894e2e0b0952c9d7c120433d1a5c215"
server_key_version: "encryptionkey@0"
I20260809 01:04:47.076328 27507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:47.092846 27507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:47.097049 27507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:47.097349 27507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:47.117158 27507 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:44035
I20260809 01:04:47.117264 27578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:44035 every 8 connection(s)
I20260809 01:04:47.118577 27507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:47.122051 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 27507
I20260809 01:04:47.122272 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:47.122882 27580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:47.122884 4426 external_mini_cluster.cc:1428] Setting key c8911ec532cee802235c51579e144744
I20260809 01:04:47.138868 27580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e: Bootstrap starting.
Aug 09 01:04:47 dist-test-slave-07c2 krb5kdc[27494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:47.143385 27580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:47.144917 27580 log.cc:824] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:47.148792 27580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e: No bootstrap required, opened a new log
I20260809 01:04:47.149729 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:47.124780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49799 (local address 127.4.82.190:44035)
0809 01:04:47.125555 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:47.125566 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:47.125607 (+ 41us) server_negotiation.cc:407] Connection header received
0809 01:04:47.127757 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:47.127778 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:47.128141 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:47.128451 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:47.130506 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:47.131923 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:47.133961 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:47.134651 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:47.134802 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:47.140227 (+ 5425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:47.140239 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:47.140260 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:47.140312 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:47.144694 (+ 4382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:47.145555 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:47.145567 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:47.145584 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:47.145828 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:47.146555 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:47.146561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:47.146564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:47.147336 (+ 772us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:47.147623 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:47.148166 (+ 543us) server_negotiation.cc:299] Negotiation successful
0809 01:04:47.148495 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":374,"threads_started":1}
I20260809 01:04:47.154021 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:47.159090 27580 raft_consensus.cc:348] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86f81a9955d4fb6bb9c951ae1386a0e" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44035 } }
I20260809 01:04:47.159622 27580 raft_consensus.cc:374] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:47.159786 27580 raft_consensus.cc:729] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c86f81a9955d4fb6bb9c951ae1386a0e, State: Initialized, Role: FOLLOWER
I20260809 01:04:47.160486 27580 consensus_queue.cc:260] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86f81a9955d4fb6bb9c951ae1386a0e" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44035 } }
I20260809 01:04:47.160825 27580 raft_consensus.cc:388] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:47.160974 27580 raft_consensus.cc:482] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:47.161154 27580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:47.164175 27580 raft_consensus.cc:504] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86f81a9955d4fb6bb9c951ae1386a0e" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44035 } }
I20260809 01:04:47.164742 27580 leader_election.cc:302] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c86f81a9955d4fb6bb9c951ae1386a0e; no voters:
I20260809 01:04:47.165406 27580 leader_election.cc:288] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:47.165550 27590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:47.167388 27590 raft_consensus.cc:686] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [term 1 LEADER]: Becoming Leader. State: Replica: c86f81a9955d4fb6bb9c951ae1386a0e, State: Running, Role: LEADER
I20260809 01:04:47.169199 27580 sys_catalog.cc:564] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:47.169736 27590 consensus_queue.cc:237] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86f81a9955d4fb6bb9c951ae1386a0e" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44035 } }
I20260809 01:04:47.175709 27592 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c86f81a9955d4fb6bb9c951ae1386a0e. Latest consensus state: current_term: 1 leader_uuid: "c86f81a9955d4fb6bb9c951ae1386a0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86f81a9955d4fb6bb9c951ae1386a0e" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44035 } } }
I20260809 01:04:47.176186 27592 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:47.176910 27591 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c86f81a9955d4fb6bb9c951ae1386a0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c86f81a9955d4fb6bb9c951ae1386a0e" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 44035 } } }
I20260809 01:04:47.177253 27591 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:47.178503 27602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:47.178761 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:47.159445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50699 (local address 127.4.82.190:44035)
0809 01:04:47.159679 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:47.159689 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:47.160203 (+ 514us) server_negotiation.cc:407] Connection header received
0809 01:04:47.160367 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:47.160371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:47.160438 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:47.160532 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:47.162540 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:47.163399 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:47.165138 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:47.165951 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:47.166133 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:47.169045 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:47.169070 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:47.169078 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:47.169120 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:47.174013 (+ 4893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:47.175122 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:47.175126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:47.175128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:47.175406 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:47.176067 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:47.176074 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:47.176077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:47.176526 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:47.177888 (+ 1362us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:47.178105 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 01:04:47.178324 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 01:04:47.184108 27602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:47.192952 27602 catalog_manager.cc:1347] Generated new cluster ID: edb97e9529eb4d169b9ef214ee47089c
I20260809 01:04:47.193055 27602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:47.215023 27602 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:47.216892 27602 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:47.237733 27602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c86f81a9955d4fb6bb9c951ae1386a0e: Generated new TSK 0
I20260809 01:04:47.238651 27602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:47.249868 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27507
I20260809 01:04:47.251587 27574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:47.344761 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 27507
W20260809 01:04:47.364687 27492 connection.cc:535] client connection to 127.4.82.190:44035 recv error: Network error: failed to read from TLS socket (remote: 127.4.82.190:44035): Connection reset by peer (error 104)
2026-08-09T01:04:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2152 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:47 dist-test-slave-07c2 krb5kdc[27614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:47 dist-test-slave-07c2 krb5kdc[27614](info): set up 2 sockets
Aug 09 01:04:47 dist-test-slave-07c2 krb5kdc[27614](info): commencing operation
krb5kdc: starting...
W20260809 01:04:48.503327 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:48 dist-test-slave-07c2 krb5kdc[27614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:04:48.793670 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41493
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:41493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:49.121899 27628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:49.122172 27628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:49.122363 27628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:49.129359 27628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:49.129480 27628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:49.129580 27628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:49.129680 27628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:49.129763 27628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:49.139535 27628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:41493
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:41493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:49.140774 27628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:49.142060 27628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:49.154177 27633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:49.154152 27634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:49.154104 27637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:49.155167 27628 server_base.cc:1034] running on GCE node
I20260809 01:04:49.155952 27628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:49.157264 27628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:49.158468 27628 hybrid_clock.cc:648] HybridClock initialized: now 1786237489158430 us; error 32 us; skew 500 ppm
I20260809 01:04:49.158818 27628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:49 dist-test-slave-07c2 krb5kdc[27614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237489, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:49.167065 27628 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:04:49.168702 27628 webserver.cc:460] Webserver started at http://127.4.82.190:46063/ using document root <none> and password file <none>
I20260809 01:04:49.169389 27628 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:49.169541 27628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:49.169849 27628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:49.172854 27628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "ba9aa8e052bb4cbaba989e3ed0ceb841"
format_stamp: "Formatted at 2026-08-09 01:04:49 on dist-test-slave-07c2"
server_key: "fe0a94c7e2a4f7156943b93f6d0f35b2"
server_key_iv: "5ed14b41b87ec7faf000725e3caf16bd"
server_key_version: "encryptionkey@0"
I20260809 01:04:49.173617 27628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "ba9aa8e052bb4cbaba989e3ed0ceb841"
format_stamp: "Formatted at 2026-08-09 01:04:49 on dist-test-slave-07c2"
server_key: "fe0a94c7e2a4f7156943b93f6d0f35b2"
server_key_iv: "5ed14b41b87ec7faf000725e3caf16bd"
server_key_version: "encryptionkey@0"
I20260809 01:04:49.179126 27628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 01:04:49.182485 27644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:49.183828 27628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:04:49.184011 27628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "ba9aa8e052bb4cbaba989e3ed0ceb841"
format_stamp: "Formatted at 2026-08-09 01:04:49 on dist-test-slave-07c2"
server_key: "fe0a94c7e2a4f7156943b93f6d0f35b2"
server_key_iv: "5ed14b41b87ec7faf000725e3caf16bd"
server_key_version: "encryptionkey@0"
I20260809 01:04:49.184213 27628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:49.200784 27628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:49.205022 27628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:49.205304 27628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:49.223968 27628 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:41493
I20260809 01:04:49.224005 27699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:41493 every 8 connection(s)
I20260809 01:04:49.225355 27628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:49.228912 27700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:49.230784 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 27628
I20260809 01:04:49.230973 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:49.231567 4426 external_mini_cluster.cc:1428] Setting key d420beedc88edd3f4369931547251f98
I20260809 01:04:49.242846 27700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841: Bootstrap starting.
I20260809 01:04:49.244591 27703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:49.233331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53843 (local address 127.4.82.190:41493)
0809 01:04:49.234135 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:49.234147 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:49.234203 (+ 56us) server_negotiation.cc:407] Connection header received
0809 01:04:49.235778 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:49.235802 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:49.236189 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:49.236500 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:49.237072 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:49.238299 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:49.240226 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:49.240887 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:49.241010 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:49.242300 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:49.242306 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:49.242319 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:49.242830 (+ 511us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:49.242907 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:49.242968 (+ 61us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:49.243110 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:49.243486 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 01:04:49.243662 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":388,"threads_started":1}
I20260809 01:04:49.246073 27700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:49.247164 27700 log.cc:824] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:49.248337 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:49.249873 27700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841: No bootstrap required, opened a new log
I20260809 01:04:49.259506 27703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:49.252738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58179 (local address 127.4.82.190:41493)
0809 01:04:49.252979 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:49.252985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:49.253351 (+ 366us) server_negotiation.cc:407] Connection header received
0809 01:04:49.253543 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:49.253547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:49.253604 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:49.253679 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:49.254289 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:49.255065 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:49.256939 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:49.257373 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:49.257473 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:49.258703 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:49.258708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:49.258711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:49.258726 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:49.258774 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:49.258789 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:49.258867 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:49.259088 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 01:04:49.259128 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 01:04:49.259469 27700 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9aa8e052bb4cbaba989e3ed0ceb841" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41493 } }
I20260809 01:04:49.259950 27700 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:49.260067 27700 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba9aa8e052bb4cbaba989e3ed0ceb841, State: Initialized, Role: FOLLOWER
I20260809 01:04:49.260720 27700 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9aa8e052bb4cbaba989e3ed0ceb841" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41493 } }
I20260809 01:04:49.260974 27700 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:49.261126 27700 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:49.261319 27700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:49.263530 27700 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9aa8e052bb4cbaba989e3ed0ceb841" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41493 } }
I20260809 01:04:49.264014 27700 leader_election.cc:302] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba9aa8e052bb4cbaba989e3ed0ceb841; no voters:
I20260809 01:04:49.264609 27700 leader_election.cc:288] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:49.264732 27710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:49.265465 27710 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [term 1 LEADER]: Becoming Leader. State: Replica: ba9aa8e052bb4cbaba989e3ed0ceb841, State: Running, Role: LEADER
I20260809 01:04:49.265964 27710 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9aa8e052bb4cbaba989e3ed0ceb841" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41493 } }
I20260809 01:04:49.266773 27700 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:49.269407 27711 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba9aa8e052bb4cbaba989e3ed0ceb841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9aa8e052bb4cbaba989e3ed0ceb841" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41493 } } }
I20260809 01:04:49.269800 27711 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:49.270166 27712 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba9aa8e052bb4cbaba989e3ed0ceb841. Latest consensus state: current_term: 1 leader_uuid: "ba9aa8e052bb4cbaba989e3ed0ceb841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba9aa8e052bb4cbaba989e3ed0ceb841" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 41493 } } }
I20260809 01:04:49.270445 27712 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:49.272027 27720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:49.277086 27720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:49.285506 27720 catalog_manager.cc:1347] Generated new cluster ID: 729ff977eae441d5b0b304ead7d1046d
I20260809 01:04:49.285619 27720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:49.293763 27720 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:49.295502 27720 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:49.303989 27720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba9aa8e052bb4cbaba989e3ed0ceb841: Generated new TSK 0
I20260809 01:04:49.304874 27720 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:49.313993 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27628
W20260809 01:04:49.315192 27695 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53843
W20260809 01:04:49.315953 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27628: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 01:04:49.316105 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 27628
W20260809 01:04:49.332559 27612 connection.cc:535] client connection to 127.4.82.190:41493 recv error: Network error: failed to read from TLS socket (remote: 127.4.82.190:41493): Connection reset by peer (error 104)
2026-08-09T01:04:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1967 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:49 dist-test-slave-07c2 krb5kdc[27732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:49 dist-test-slave-07c2 krb5kdc[27732](info): set up 2 sockets
Aug 09 01:04:49 dist-test-slave-07c2 krb5kdc[27732](info): commencing operation
krb5kdc: starting...
W20260809 01:04:50.443437 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.010s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:50 dist-test-slave-07c2 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:04:50.754945 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:45917
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:45917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:51.088363 27745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:51.088696 27745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:51.088864 27745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:51.095965 27745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:51.096122 27745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:51.096233 27745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:51.096349 27745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:51.096424 27745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:51.106003 27745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:45917
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:45917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:51.107321 27745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:51.108634 27745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:51.120688 27755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:51.120645 27752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:51.120731 27751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:51.124562 27745 server_base.cc:1034] running on GCE node
I20260809 01:04:51.125337 27745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:51.126660 27745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:51.127862 27745 hybrid_clock.cc:648] HybridClock initialized: now 1786237491127826 us; error 32 us; skew 500 ppm
I20260809 01:04:51.128196 27745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:51 dist-test-slave-07c2 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:51.131244 27745 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:04:51.132802 27745 webserver.cc:460] Webserver started at http://127.4.82.190:37795/ using document root <none> and password file <none>
I20260809 01:04:51.133538 27745 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:51.133689 27745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:51.134001 27745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:51.136981 27745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "b39250db9f584a78928b976d057ffa5d"
format_stamp: "Formatted at 2026-08-09 01:04:51 on dist-test-slave-07c2"
server_key: "31278e09ecb1058d164bdd614faedc9c"
server_key_iv: "872b2d86e6de8a58b17cfad00f2e9e07"
server_key_version: "encryptionkey@0"
I20260809 01:04:51.137737 27745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "b39250db9f584a78928b976d057ffa5d"
format_stamp: "Formatted at 2026-08-09 01:04:51 on dist-test-slave-07c2"
server_key: "31278e09ecb1058d164bdd614faedc9c"
server_key_iv: "872b2d86e6de8a58b17cfad00f2e9e07"
server_key_version: "encryptionkey@0"
I20260809 01:04:51.142647 27745 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 01:04:51.146102 27764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:51.147425 27745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:04:51.147591 27745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "b39250db9f584a78928b976d057ffa5d"
format_stamp: "Formatted at 2026-08-09 01:04:51 on dist-test-slave-07c2"
server_key: "31278e09ecb1058d164bdd614faedc9c"
server_key_iv: "872b2d86e6de8a58b17cfad00f2e9e07"
server_key_version: "encryptionkey@0"
I20260809 01:04:51.147785 27745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:51.153497 27745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:51.157243 27745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:51.157528 27745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:51.176239 27745 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:45917
I20260809 01:04:51.176288 27819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:45917 every 8 connection(s)
I20260809 01:04:51.177626 27745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:51.181351 27820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:51.184039 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 27745
I20260809 01:04:51.184211 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:51.184672 4426 external_mini_cluster.cc:1428] Setting key 1b0da423c69b2fa73c61f74b6584f6b6
I20260809 01:04:51.192412 27823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:51.186511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46897 (local address 127.4.82.190:45917)
0809 01:04:51.187418 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:51.187429 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:51.187468 (+ 39us) server_negotiation.cc:407] Connection header received
0809 01:04:51.189107 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:51.189127 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:51.189453 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:51.189752 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:51.190856 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:51.190862 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:51.190878 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:51.190895 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:51.190950 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:51.190971 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:51.191133 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:51.191361 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 01:04:51.191563 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":413,"threads_started":1}
I20260809 01:04:51.193043 27820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d: Bootstrap starting.
I20260809 01:04:51.195824 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:51.196388 27820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:51.197597 27820 log.cc:824] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:51.200875 27820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d: No bootstrap required, opened a new log
I20260809 01:04:51.203466 27823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:51.200919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58405 (local address 127.4.82.190:45917)
0809 01:04:51.201078 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:51.201085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:51.201587 (+ 502us) server_negotiation.cc:407] Connection header received
0809 01:04:51.201706 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:51.201711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:51.201764 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:51.201861 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:51.202766 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:51.202778 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:51.202782 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:51.202807 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:51.202858 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:51.202883 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:51.203000 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:51.203070 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 01:04:51.203135 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 01:04:51.209151 27820 raft_consensus.cc:348] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b39250db9f584a78928b976d057ffa5d" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 45917 } }
I20260809 01:04:51.209513 27820 raft_consensus.cc:374] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:51.209649 27820 raft_consensus.cc:729] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b39250db9f584a78928b976d057ffa5d, State: Initialized, Role: FOLLOWER
I20260809 01:04:51.210299 27820 consensus_queue.cc:260] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b39250db9f584a78928b976d057ffa5d" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 45917 } }
I20260809 01:04:51.210557 27820 raft_consensus.cc:388] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:51.210718 27820 raft_consensus.cc:482] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:51.210889 27820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:51.213068 27820 raft_consensus.cc:504] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b39250db9f584a78928b976d057ffa5d" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 45917 } }
I20260809 01:04:51.213553 27820 leader_election.cc:302] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b39250db9f584a78928b976d057ffa5d; no voters:
I20260809 01:04:51.214159 27820 leader_election.cc:288] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:51.214267 27830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:51.215020 27830 raft_consensus.cc:686] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [term 1 LEADER]: Becoming Leader. State: Replica: b39250db9f584a78928b976d057ffa5d, State: Running, Role: LEADER
I20260809 01:04:51.215577 27830 consensus_queue.cc:237] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b39250db9f584a78928b976d057ffa5d" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 45917 } }
I20260809 01:04:51.216223 27820 sys_catalog.cc:564] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:51.218587 27832 sys_catalog.cc:455] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b39250db9f584a78928b976d057ffa5d. Latest consensus state: current_term: 1 leader_uuid: "b39250db9f584a78928b976d057ffa5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b39250db9f584a78928b976d057ffa5d" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 45917 } } }
I20260809 01:04:51.219038 27832 sys_catalog.cc:458] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:51.219548 27831 sys_catalog.cc:455] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b39250db9f584a78928b976d057ffa5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b39250db9f584a78928b976d057ffa5d" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 45917 } } }
I20260809 01:04:51.219971 27831 sys_catalog.cc:458] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:51.221321 27835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:51.226449 27835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:51.234445 27835 catalog_manager.cc:1347] Generated new cluster ID: 574323eb9f41435ea74aedfd96176c75
I20260809 01:04:51.234562 27835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:51.253047 27835 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:51.253520 27835 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:51.259500 27835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b39250db9f584a78928b976d057ffa5d: Generated new TSK 0
I20260809 01:04:51.260214 27835 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:51.270004 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27745
W20260809 01:04:51.271116 27814 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46897
W20260809 01:04:51.271956 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27745: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 01:04:51.272119 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 27745
2026-08-09T01:04:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1959 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:51 dist-test-slave-07c2 krb5kdc[27851](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:51 dist-test-slave-07c2 krb5kdc[27851](info): set up 2 sockets
Aug 09 01:04:51 dist-test-slave-07c2 krb5kdc[27851](info): commencing operation
krb5kdc: starting...
W20260809 01:04:52.419049 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:52 dist-test-slave-07c2 krb5kdc[27851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:04:52.710445 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:46523
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:41359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:46523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:53.040222 27866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:53.040494 27866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:53.040700 27866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:53.047839 27866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:53.047964 27866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:53.048076 27866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:53.048189 27866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:53.048275 27866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:53.057789 27866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:41359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:46523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:46523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:53.059000 27866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:53.060295 27866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:53.072124 27872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:53.072173 27875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:53.072247 27873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:53.076610 27866 server_base.cc:1034] running on GCE node
I20260809 01:04:53.077308 27866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:53.078544 27866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:53.079739 27866 hybrid_clock.cc:648] HybridClock initialized: now 1786237493079696 us; error 44 us; skew 500 ppm
I20260809 01:04:53.080094 27866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:53 dist-test-slave-07c2 krb5kdc[27851](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237493, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:53.083107 27866 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:04:53.084636 27866 webserver.cc:460] Webserver started at http://127.4.82.190:44193/ using document root <none> and password file <none>
I20260809 01:04:53.085321 27866 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:53.085456 27866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:53.085767 27866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:53.088706 27866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "bb6d3cf1376b4b159baf37003cdc6b11"
format_stamp: "Formatted at 2026-08-09 01:04:53 on dist-test-slave-07c2"
server_key: "f17a4696e1da4ef41eb8d1f183899d7b"
server_key_iv: "a40a799061ff047bf2f704f529db4816"
server_key_version: "encryptionkey@0"
I20260809 01:04:53.089466 27866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "bb6d3cf1376b4b159baf37003cdc6b11"
format_stamp: "Formatted at 2026-08-09 01:04:53 on dist-test-slave-07c2"
server_key: "f17a4696e1da4ef41eb8d1f183899d7b"
server_key_iv: "a40a799061ff047bf2f704f529db4816"
server_key_version: "encryptionkey@0"
I20260809 01:04:53.094360 27866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 01:04:53.097641 27883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:53.098927 27866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 01:04:53.099105 27866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "bb6d3cf1376b4b159baf37003cdc6b11"
format_stamp: "Formatted at 2026-08-09 01:04:53 on dist-test-slave-07c2"
server_key: "f17a4696e1da4ef41eb8d1f183899d7b"
server_key_iv: "a40a799061ff047bf2f704f529db4816"
server_key_version: "encryptionkey@0"
I20260809 01:04:53.099330 27866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:53.123227 27866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:53.127059 27866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:53.127403 27866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:53.146453 27866 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:46523
I20260809 01:04:53.146492 27940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:46523 every 8 connection(s)
I20260809 01:04:53.147969 27866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:53.152670 27941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:53.158164 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 27866
I20260809 01:04:53.158370 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:53.158847 4426 external_mini_cluster.cc:1428] Setting key db506cbccbf064de3492fbdba9a3b751
I20260809 01:04:53.165017 27941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11: Bootstrap starting.
I20260809 01:04:53.168998 27941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:53.170069 27941 log.cc:824] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:53.173192 27941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11: No bootstrap required, opened a new log
Aug 09 01:04:53 dist-test-slave-07c2 krb5kdc[27851](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:53.183197 27941 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6d3cf1376b4b159baf37003cdc6b11" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46523 } }
I20260809 01:04:53.183662 27941 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:53.183799 27941 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb6d3cf1376b4b159baf37003cdc6b11, State: Initialized, Role: FOLLOWER
I20260809 01:04:53.184533 27945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:53.160896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50797 (local address 127.4.82.190:46523)
0809 01:04:53.161885 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:53.161902 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:53.161965 (+ 63us) server_negotiation.cc:407] Connection header received
0809 01:04:53.164429 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:53.164459 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:53.165012 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:53.165484 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:53.166836 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:53.168927 (+ 2091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:53.170707 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:53.171557 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:53.171727 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:53.175916 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:53.175925 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:53.175944 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:53.175996 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:53.180063 (+ 4067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:53.180733 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:53.180746 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:53.180754 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:53.180983 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:53.181332 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:53.181339 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:53.181348 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:53.182042 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:53.182338 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:53.182715 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 01:04:53.183121 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":457,"threads_started":1}
I20260809 01:04:53.184433 27941 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6d3cf1376b4b159baf37003cdc6b11" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46523 } }
I20260809 01:04:53.184729 27941 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:53.184883 27941 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:53.185066 27941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:53.187919 27941 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6d3cf1376b4b159baf37003cdc6b11" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46523 } }
I20260809 01:04:53.188299 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:53.188450 27941 leader_election.cc:302] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb6d3cf1376b4b159baf37003cdc6b11; no voters:
I20260809 01:04:53.189052 27941 leader_election.cc:288] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:53.189209 27947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:53.190943 27941 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:53.196893 27947 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [term 1 LEADER]: Becoming Leader. State: Replica: bb6d3cf1376b4b159baf37003cdc6b11, State: Running, Role: LEADER
I20260809 01:04:53.197645 27947 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6d3cf1376b4b159baf37003cdc6b11" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46523 } }
I20260809 01:04:53.201206 27948 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb6d3cf1376b4b159baf37003cdc6b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6d3cf1376b4b159baf37003cdc6b11" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46523 } } }
I20260809 01:04:53.201579 27948 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:53.202091 27949 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb6d3cf1376b4b159baf37003cdc6b11. Latest consensus state: current_term: 1 leader_uuid: "bb6d3cf1376b4b159baf37003cdc6b11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb6d3cf1376b4b159baf37003cdc6b11" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 46523 } } }
I20260809 01:04:53.202453 27949 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:53.203778 27967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:53.210377 27967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:53.218657 27945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:53.198557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54173 (local address 127.4.82.190:46523)
0809 01:04:53.198763 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:53.198772 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:53.199786 (+ 1014us) server_negotiation.cc:407] Connection header received
0809 01:04:53.200042 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:53.200047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:53.200121 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:53.200234 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:53.202706 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:53.204259 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:53.206850 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:53.207862 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:53.208014 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:53.211682 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:53.211687 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:53.211690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:53.211725 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:53.214892 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:53.215798 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:53.215803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:53.215806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:53.215961 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:53.216395 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:53.216400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:53.216403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:53.216846 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:53.217112 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:53.218264 (+ 1152us) server_negotiation.cc:299] Negotiation successful
0809 01:04:53.218413 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 01:04:53.220242 27967 catalog_manager.cc:1347] Generated new cluster ID: 3dfbbe3a2d8f415d984ad7064e25e615
I20260809 01:04:53.220351 27967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:53.233997 27967 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:53.235759 27967 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:53.244907 27967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb6d3cf1376b4b159baf37003cdc6b11: Generated new TSK 0
I20260809 01:04:53.245851 27967 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:53.253937 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27866
I20260809 01:04:53.255527 27935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:04:53.344079 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 27866
2026-08-09T01:04:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2072 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:53 dist-test-slave-07c2 krb5kdc[27975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:53 dist-test-slave-07c2 krb5kdc[27975](info): set up 2 sockets
Aug 09 01:04:53 dist-test-slave-07c2 krb5kdc[27975](info): commencing operation
krb5kdc: starting...
W20260809 01:04:54.487247 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.007s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:54 dist-test-slave-07c2 krb5kdc[27975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:04:54.794549 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:39529
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:43947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:39529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:55.132303 27990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:55.132562 27990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:55.132762 27990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:55.139720 27990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:55.139815 27990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:55.139945 27990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:55.140055 27990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:55.140141 27990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:55.149627 27990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:43947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:39529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:39529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:55.150840 27990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:55.152127 27990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:55.163708 27995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:55.163774 27997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:55.163815 27999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:55.164587 27990 server_base.cc:1034] running on GCE node
I20260809 01:04:55.165323 27990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:55.166685 27990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:55.167914 27990 hybrid_clock.cc:648] HybridClock initialized: now 1786237495167861 us; error 40 us; skew 500 ppm
I20260809 01:04:55.168243 27990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:55 dist-test-slave-07c2 krb5kdc[27975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237495, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:55.176183 27990 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:04:55.177723 27990 webserver.cc:460] Webserver started at http://127.4.82.190:39597/ using document root <none> and password file <none>
I20260809 01:04:55.178387 27990 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:55.178535 27990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:55.178848 27990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:55.181807 27990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "2ae4c9cc8f274ec3a43817a02557a02f"
format_stamp: "Formatted at 2026-08-09 01:04:55 on dist-test-slave-07c2"
server_key: "089cd70679063c97226fda4ee4beada6"
server_key_iv: "c0d01243e82a9dbf348f3505e37bffb4"
server_key_version: "encryptionkey@0"
I20260809 01:04:55.182546 27990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "2ae4c9cc8f274ec3a43817a02557a02f"
format_stamp: "Formatted at 2026-08-09 01:04:55 on dist-test-slave-07c2"
server_key: "089cd70679063c97226fda4ee4beada6"
server_key_iv: "c0d01243e82a9dbf348f3505e37bffb4"
server_key_version: "encryptionkey@0"
I20260809 01:04:55.187491 27990 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 01:04:55.190698 28007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:55.191959 27990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:04:55.192142 27990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "2ae4c9cc8f274ec3a43817a02557a02f"
format_stamp: "Formatted at 2026-08-09 01:04:55 on dist-test-slave-07c2"
server_key: "089cd70679063c97226fda4ee4beada6"
server_key_iv: "c0d01243e82a9dbf348f3505e37bffb4"
server_key_version: "encryptionkey@0"
I20260809 01:04:55.192333 27990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:55.210753 27990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:55.214944 27990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:55.215242 27990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:55.233878 27990 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:39529
I20260809 01:04:55.233938 28060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:39529 every 8 connection(s)
I20260809 01:04:55.235208 27990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:55.238730 28061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:55.244158 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 27990
I20260809 01:04:55.244338 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:55.244951 4426 external_mini_cluster.cc:1428] Setting key 22b6fd2c532c16bd0845f064ce94878c
I20260809 01:04:55.249684 28061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f: Bootstrap starting.
I20260809 01:04:55.253239 28061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:55.254371 28061 log.cc:824] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:55.257560 28061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f: No bootstrap required, opened a new log
I20260809 01:04:55.258343 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:55.246736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37285 (local address 127.4.82.190:39529)
0809 01:04:55.247615 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:55.247625 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:55.247666 (+ 41us) server_negotiation.cc:407] Connection header received
0809 01:04:55.249320 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:55.249341 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:55.249694 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:55.250062 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:55.250637 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:55.252030 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:55.253885 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:55.254582 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:55.254731 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:55.256160 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:55.256167 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:55.256179 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:55.256671 (+ 492us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:55.256750 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:55.256774 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:55.256859 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:55.257096 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 01:04:55.257346 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":418,"threads_started":1}
I20260809 01:04:55.261947 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:55.265179 28061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae4c9cc8f274ec3a43817a02557a02f" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 39529 } }
I20260809 01:04:55.265533 28061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:55.265641 28061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ae4c9cc8f274ec3a43817a02557a02f, State: Initialized, Role: FOLLOWER
I20260809 01:04:55.266326 28061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae4c9cc8f274ec3a43817a02557a02f" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 39529 } }
I20260809 01:04:55.266681 28061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:55.266837 28061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:55.267037 28061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:55.269357 28061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae4c9cc8f274ec3a43817a02557a02f" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 39529 } }
I20260809 01:04:55.269863 28061 leader_election.cc:302] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ae4c9cc8f274ec3a43817a02557a02f; no voters:
I20260809 01:04:55.270565 28061 leader_election.cc:288] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:55.270676 28071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:55.272732 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:55.266347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42923 (local address 127.4.82.190:39529)
0809 01:04:55.266545 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:55.266552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:55.266982 (+ 430us) server_negotiation.cc:407] Connection header received
0809 01:04:55.267168 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:55.267173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:55.267247 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:55.267407 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:55.267945 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:55.268689 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:55.270062 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:55.270546 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:55.270689 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:55.271979 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:55.271985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:55.271990 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:55.272021 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:55.272057 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:55.272073 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:55.272123 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:55.272320 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 01:04:55.272407 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 01:04:55.273213 28061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:55.275599 28071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [term 1 LEADER]: Becoming Leader. State: Replica: 2ae4c9cc8f274ec3a43817a02557a02f, State: Running, Role: LEADER
I20260809 01:04:55.276330 28071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae4c9cc8f274ec3a43817a02557a02f" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 39529 } }
I20260809 01:04:55.279609 28072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ae4c9cc8f274ec3a43817a02557a02f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae4c9cc8f274ec3a43817a02557a02f" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 39529 } } }
I20260809 01:04:55.280005 28072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:55.279933 28073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ae4c9cc8f274ec3a43817a02557a02f. Latest consensus state: current_term: 1 leader_uuid: "2ae4c9cc8f274ec3a43817a02557a02f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ae4c9cc8f274ec3a43817a02557a02f" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 39529 } } }
I20260809 01:04:55.280238 28073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:55.283752 28083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:55.288897 28083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:55.297457 28083 catalog_manager.cc:1347] Generated new cluster ID: 569f5e7e374a47e2935eefd7ab64367f
I20260809 01:04:55.297578 28083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:55.326141 28083 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:55.327483 28083 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:55.345817 28083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ae4c9cc8f274ec3a43817a02557a02f: Generated new TSK 0
I20260809 01:04:55.346613 28083 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:55.351976 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27990
W20260809 01:04:55.353220 28056 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37285
W20260809 01:04:55.353897 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 27990: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 01:04:55.354049 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 27990
2026-08-09T01:04:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2009 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:55 dist-test-slave-07c2 krb5kdc[28094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:55 dist-test-slave-07c2 krb5kdc[28094](info): set up 2 sockets
Aug 09 01:04:55 dist-test-slave-07c2 krb5kdc[28094](info): commencing operation
krb5kdc: starting...
W20260809 01:04:56.512321 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:56 dist-test-slave-07c2 krb5kdc[28094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:04:56.818513 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:37081
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:37081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:57.150198 28108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:57.150550 28108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:57.150740 28108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:57.157927 28108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:57.158097 28108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:57.158218 28108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:57.158331 28108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:57.158422 28108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:57.168246 28108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:37081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:37081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:57.169620 28108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:57.170964 28108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:57.183035 28115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:57.183131 28114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:57.183153 28118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:57.183854 28108 server_base.cc:1034] running on GCE node
I20260809 01:04:57.184712 28108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:57.186126 28108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:57.187388 28108 hybrid_clock.cc:648] HybridClock initialized: now 1786237497187346 us; error 58 us; skew 500 ppm
I20260809 01:04:57.187759 28108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:57 dist-test-slave-07c2 krb5kdc[28094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237497, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:57.190825 28108 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:04:57.192350 28108 webserver.cc:460] Webserver started at http://127.4.82.190:33073/ using document root <none> and password file <none>
I20260809 01:04:57.193078 28108 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:57.193234 28108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:57.193547 28108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:57.196605 28108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "3635833f26c84cf4b517ac0092dfd3d9"
format_stamp: "Formatted at 2026-08-09 01:04:57 on dist-test-slave-07c2"
server_key: "795257de5f7c6ee01624b750b560f6f0"
server_key_iv: "4f7fe00207fcbdfac08ed4229ce8f516"
server_key_version: "encryptionkey@0"
I20260809 01:04:57.197402 28108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "3635833f26c84cf4b517ac0092dfd3d9"
format_stamp: "Formatted at 2026-08-09 01:04:57 on dist-test-slave-07c2"
server_key: "795257de5f7c6ee01624b750b560f6f0"
server_key_iv: "4f7fe00207fcbdfac08ed4229ce8f516"
server_key_version: "encryptionkey@0"
I20260809 01:04:57.203042 28108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 01:04:57.206486 28125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:57.207861 28108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 01:04:57.208053 28108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "3635833f26c84cf4b517ac0092dfd3d9"
format_stamp: "Formatted at 2026-08-09 01:04:57 on dist-test-slave-07c2"
server_key: "795257de5f7c6ee01624b750b560f6f0"
server_key_iv: "4f7fe00207fcbdfac08ed4229ce8f516"
server_key_version: "encryptionkey@0"
I20260809 01:04:57.208263 28108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:57.213970 28108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:57.217857 28108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:57.218153 28108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:57.236776 28108 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:37081
I20260809 01:04:57.236819 28178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:37081 every 8 connection(s)
I20260809 01:04:57.238106 28108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:57.241581 28179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:57.245362 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 28108
I20260809 01:04:57.245546 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:57.246097 4426 external_mini_cluster.cc:1428] Setting key 53787df4755644ca3c0e9d7a9f4adcda
I20260809 01:04:57.253515 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:57.247877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35669 (local address 127.4.82.190:37081)
0809 01:04:57.248665 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:57.248676 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:57.248717 (+ 41us) server_negotiation.cc:407] Connection header received
0809 01:04:57.250269 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:57.250289 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:57.250613 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:57.250924 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:57.252012 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:57.252018 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:57.252032 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:57.252047 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:57.252103 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:57.252124 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:57.252308 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:57.252445 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 01:04:57.252633 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":369,"threads_started":1}
I20260809 01:04:57.254344 28179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9: Bootstrap starting.
I20260809 01:04:57.257079 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:04:57.257674 28179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9: Neither blocks nor log segments found. Creating new log.
I20260809 01:04:57.258625 28179 log.cc:824] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9: Log is configured to *not* fsync() on all Append() calls
I20260809 01:04:57.261401 28179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9: No bootstrap required, opened a new log
I20260809 01:04:57.263998 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:57.261564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33741 (local address 127.4.82.190:37081)
0809 01:04:57.261748 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:57.261754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:57.262106 (+ 352us) server_negotiation.cc:407] Connection header received
0809 01:04:57.262274 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:57.262277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:57.262317 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:57.262391 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:57.263364 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:57.263368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:57.263371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 01:04:57.263385 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:57.263421 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 01:04:57.263435 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:57.263532 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:57.263655 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 01:04:57.263700 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 01:04:57.269345 28179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3635833f26c84cf4b517ac0092dfd3d9" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37081 } }
I20260809 01:04:57.269686 28179 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 01:04:57.269804 28179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3635833f26c84cf4b517ac0092dfd3d9, State: Initialized, Role: FOLLOWER
I20260809 01:04:57.270383 28179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3635833f26c84cf4b517ac0092dfd3d9" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37081 } }
I20260809 01:04:57.270618 28179 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 01:04:57.270782 28179 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 01:04:57.270959 28179 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:04:57.273057 28179 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3635833f26c84cf4b517ac0092dfd3d9" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37081 } }
I20260809 01:04:57.273518 28179 leader_election.cc:302] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3635833f26c84cf4b517ac0092dfd3d9; no voters:
I20260809 01:04:57.274083 28179 leader_election.cc:288] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 01:04:57.274220 28189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:04:57.274933 28189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [term 1 LEADER]: Becoming Leader. State: Replica: 3635833f26c84cf4b517ac0092dfd3d9, State: Running, Role: LEADER
I20260809 01:04:57.275498 28189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3635833f26c84cf4b517ac0092dfd3d9" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37081 } }
I20260809 01:04:57.276084 28179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:04:57.279313 28191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3635833f26c84cf4b517ac0092dfd3d9. Latest consensus state: current_term: 1 leader_uuid: "3635833f26c84cf4b517ac0092dfd3d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3635833f26c84cf4b517ac0092dfd3d9" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37081 } } }
I20260809 01:04:57.279279 28190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3635833f26c84cf4b517ac0092dfd3d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3635833f26c84cf4b517ac0092dfd3d9" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 37081 } } }
I20260809 01:04:57.279809 28191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:57.279816 28190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9 [sys.catalog]: This master's current role is: LEADER
I20260809 01:04:57.281198 28196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:04:57.285811 28196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:04:57.294077 28196 catalog_manager.cc:1347] Generated new cluster ID: 9aaeab201fd9495283ced9941e96a828
I20260809 01:04:57.294173 28196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:04:57.328440 28196 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:04:57.328912 28196 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:04:57.334519 28196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3635833f26c84cf4b517ac0092dfd3d9: Generated new TSK 0
I20260809 01:04:57.335371 28196 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:04:57.349062 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 28108
W20260809 01:04:57.350152 28174 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35669
W20260809 01:04:57.350899 4426 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 28108: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 01:04:57.351037 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 28108
2026-08-09T01:04:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1995 ms)
[----------] 18 tests from AuthTokenIssuingTest (36820 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:04:57 dist-test-slave-07c2 krb5kdc[28211](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:04:57 dist-test-slave-07c2 krb5kdc[28211](info): set up 2 sockets
Aug 09 01:04:57 dist-test-slave-07c2 krb5kdc[28211](info): commencing operation
krb5kdc: starting...
W20260809 01:04:58.492118 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.003s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:04:58 dist-test-slave-07c2 krb5kdc[28211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:04:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:04:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:04:58.778478 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:38459
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:38459,127.4.82.189:42171,127.4.82.188:42243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:04:59.103065 28224 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 01:04:59.103453 28224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:59.103582 28224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:59.103708 28224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:04:59.110756 28224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:59.110833 28224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:59.110957 28224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:59.111052 28224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:59.111150 28224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:59.121387 28224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:38459,127.4.82.189:42171,127.4.82.188:42243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:38459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:59.122555 28224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:59.123917 28224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:59.135406 28232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:59.135456 28234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:59.135514 28231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:59.136396 28224 server_base.cc:1034] running on GCE node
I20260809 01:04:59.137112 28224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:59.138429 28224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:59.139667 28224 hybrid_clock.cc:648] HybridClock initialized: now 1786237499139597 us; error 49 us; skew 500 ppm
I20260809 01:04:59.139978 28224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:59 dist-test-slave-07c2 krb5kdc[28211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:59.143139 28224 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:04:59.144531 28224 webserver.cc:460] Webserver started at http://127.4.82.190:33039/ using document root <none> and password file <none>
I20260809 01:04:59.145205 28224 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:59.145334 28224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:59.145646 28224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:59.148563 28224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "7ba1371b42594538bb0c15fff07acdd4"
format_stamp: "Formatted at 2026-08-09 01:04:59 on dist-test-slave-07c2"
server_key: "fa0b1a652a838428ff5a28b14cf0a380"
server_key_iv: "fc7ca1f547064ab9d3fb2e6b5cabbbe0"
server_key_version: "encryptionkey@0"
I20260809 01:04:59.149328 28224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "7ba1371b42594538bb0c15fff07acdd4"
format_stamp: "Formatted at 2026-08-09 01:04:59 on dist-test-slave-07c2"
server_key: "fa0b1a652a838428ff5a28b14cf0a380"
server_key_iv: "fc7ca1f547064ab9d3fb2e6b5cabbbe0"
server_key_version: "encryptionkey@0"
I20260809 01:04:59.154206 28224 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 01:04:59.157505 28241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:59.158758 28224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 01:04:59.158936 28224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "7ba1371b42594538bb0c15fff07acdd4"
format_stamp: "Formatted at 2026-08-09 01:04:59 on dist-test-slave-07c2"
server_key: "fa0b1a652a838428ff5a28b14cf0a380"
server_key_iv: "fc7ca1f547064ab9d3fb2e6b5cabbbe0"
server_key_version: "encryptionkey@0"
I20260809 01:04:59.159140 28224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:59.176242 28224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:59.179931 28224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:59.180182 28224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:59.198436 28224 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:38459
I20260809 01:04:59.198470 28295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:38459 every 8 connection(s)
I20260809 01:04:59.199726 28224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:04:59.202404 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 28224
I20260809 01:04:59.202638 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:04:59.203482 4426 external_mini_cluster.cc:1428] Setting key d021304f00a9ae02d570029b66da89aa
I20260809 01:04:59.203586 28296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:59.213136 28296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:04:59 dist-test-slave-07c2 krb5kdc[28211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.82.189@KRBTEST.COM; defaulting to no policy
I20260809 01:04:59.238066 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.216788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59250 (local address 127.4.82.190:38459)
0809 01:04:59.217773 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:59.217787 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:59.218283 (+ 496us) server_negotiation.cc:407] Connection header received
0809 01:04:59.220436 (+ 2153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:59.220446 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:59.220734 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:59.220943 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:59.222037 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.224074 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.225449 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.226401 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.226702 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:59.229935 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:59.229953 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:59.229972 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:59.230027 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:59.234010 (+ 3983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:59.234674 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:59.234684 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:59.234688 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:59.234897 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:59.235322 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:59.235329 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:59.235335 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:59.236031 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:59.236227 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:59.236792 (+ 565us) server_negotiation.cc:299] Negotiation successful
0809 01:04:59.237123 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":597,"thread_start_us":379,"threads_started":1}
I20260809 01:04:59.238130 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.216780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:38459 (local address 127.0.0.1:59250)
0809 01:04:59.218101 (+ 1321us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.218156 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:59.220093 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:59.220961 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:59.220967 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:59.221493 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:59.221822 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:59.221839 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.224249 (+ 2410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.224253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:59.225304 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:59.225316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.226615 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.226620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:59.226768 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:59.227508 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:04:59.227529 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:59.229686 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:59.234151 (+ 4465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:59.234172 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:59.234179 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:59.234538 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:59.235010 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:59.235013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:59.235015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:59.235182 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:59.236181 (+ 999us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:59.236187 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:59.236380 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 01:04:59.236587 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 01:04:59.236959 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1048,"mutex_wait_us":41,"spinlock_wait_cycles":19840,"thread_start_us":848,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.189@KRBTEST.COM" created.
I20260809 01:04:59.240957 28296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:04:59.242089 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.241737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:42171 (local address 127.0.0.1:35476)
0809 01:04:59.241891 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.241991 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 01:04:59.242354 28245 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
I20260809 01:04:59.243885 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.243540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:42171 (local address 127.0.0.1:35480)
0809 01:04:59.243708 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.243771 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 01:04:59.244077 28296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:42171: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.189.keytab.
Entry for principal kudu/127.4.82.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.189.keytab.
I20260809 01:04:59.272411 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.189:42171
--webserver_interface=127.4.82.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.189.keytab
--principal=kudu/127.4.82.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:38459,127.4.82.189:42171,127.4.82.188:42243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:04:59.293503 28296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } attempt: 1
I20260809 01:04:59.294603 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.294174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:42171 (local address 127.0.0.1:35494)
0809 01:04:59.294372 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.294480 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 01:04:59.295606 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.295225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:42171 (local address 127.0.0.1:35508)
0809 01:04:59.295390 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.295460 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 01:04:59.295827 28296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:42171: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
I20260809 01:04:59.364044 28296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } attempt: 2
I20260809 01:04:59.365250 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.364682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:42171 (local address 127.0.0.1:35514)
0809 01:04:59.364932 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.365057 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 01:04:59.366396 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.365931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:42171 (local address 127.0.0.1:35516)
0809 01:04:59.366087 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.366200 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 01:04:59.366595 28296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:42171: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
I20260809 01:04:59.457881 28296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } attempt: 3
I20260809 01:04:59.459414 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.458699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:42171 (local address 127.0.0.1:35530)
0809 01:04:59.458985 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.459145 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260809 01:04:59.460597 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.460110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:42171 (local address 127.0.0.1:35546)
0809 01:04:59.460308 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.460406 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 01:04:59.460851 28296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:42171: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
I20260809 01:04:59.604115 28296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } attempt: 4
W20260809 01:04:59.605029 28303 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
I20260809 01:04:59.605379 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.604745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:42171 (local address 127.0.0.1:35552)
0809 01:04:59.605084 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.605213 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
W20260809 01:04:59.605464 28303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:04:59.605624 28303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:04:59.605773 28303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 01:04:59.606458 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.606067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:42171 (local address 127.0.0.1:35554)
0809 01:04:59.606217 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.606320 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 01:04:59.606630 28296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:42171: Network error: Client connection negotiation failed: client connection to 127.4.82.189:42171: connect: Connection refused (error 111)
W20260809 01:04:59.613219 28303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:04:59.613293 28303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:04:59.613348 28303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:04:59.613417 28303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:04:59.613517 28303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:04:59.623107 28303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:38459,127.4.82.189:42171,127.4.82.188:42243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.189.keytab
--principal=kudu/127.4.82.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.189:42171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.82.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:04:59.624359 28303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:04:59.625603 28303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:04:59.637236 28310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:59.637195 28312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:04:59.637463 28309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:04:59.637863 28303 server_base.cc:1034] running on GCE node
I20260809 01:04:59.638617 28303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:04:59.639966 28303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:04:59.641202 28303 hybrid_clock.cc:648] HybridClock initialized: now 1786237499641125 us; error 58 us; skew 500 ppm
I20260809 01:04:59.641508 28303 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:04:59 dist-test-slave-07c2 krb5kdc[28211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:04:59.644490 28303 init.cc:377] Logged in from keytab as kudu/127.4.82.189@KRBTEST.COM (short username kudu)
I20260809 01:04:59.645794 28303 webserver.cc:460] Webserver started at http://127.4.82.189:35679/ using document root <none> and password file <none>
I20260809 01:04:59.646447 28303 fs_manager.cc:359] Metadata directory not provided
I20260809 01:04:59.646582 28303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:04:59.646896 28303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:04:59.649777 28303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/data/instance:
uuid: "c5a7573f0aad4d7b837e25a3491b49cd"
format_stamp: "Formatted at 2026-08-09 01:04:59 on dist-test-slave-07c2"
server_key: "10a05bfd67705a94f2fb8ec94c13fd68"
server_key_iv: "2dcc43a22b9c511d7ed32f83348ce4fe"
server_key_version: "encryptionkey@0"
I20260809 01:04:59.650513 28303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/wal/instance:
uuid: "c5a7573f0aad4d7b837e25a3491b49cd"
format_stamp: "Formatted at 2026-08-09 01:04:59 on dist-test-slave-07c2"
server_key: "10a05bfd67705a94f2fb8ec94c13fd68"
server_key_iv: "2dcc43a22b9c511d7ed32f83348ce4fe"
server_key_version: "encryptionkey@0"
I20260809 01:04:59.655835 28303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 01:04:59.658901 28319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:04:59.660143 28303 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 01:04:59.660328 28303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/wal
uuid: "c5a7573f0aad4d7b837e25a3491b49cd"
format_stamp: "Formatted at 2026-08-09 01:04:59 on dist-test-slave-07c2"
server_key: "10a05bfd67705a94f2fb8ec94c13fd68"
server_key_iv: "2dcc43a22b9c511d7ed32f83348ce4fe"
server_key_version: "encryptionkey@0"
I20260809 01:04:59.660533 28303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:04:59.690827 28303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:04:59.694651 28303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:04:59.694959 28303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:04:59.712896 28303 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.189:42171
I20260809 01:04:59.712930 28372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.189:42171 every 8 connection(s)
I20260809 01:04:59.714161 28303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/data/info.pb
I20260809 01:04:59.717669 28373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:04:59.718351 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 28303
I20260809 01:04:59.718554 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-1/wal/instance
I20260809 01:04:59.719137 4426 external_mini_cluster.cc:1428] Setting key 3a8a71d74d5a70bed8d1a4e36639d742
I20260809 01:04:59.727388 28373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:04:59 dist-test-slave-07c2 krb5kdc[28211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.189@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:04:59.748144 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.730585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59256 (local address 127.4.82.190:38459)
0809 01:04:59.730807 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:59.730814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:59.732109 (+ 1295us) server_negotiation.cc:407] Connection header received
0809 01:04:59.733884 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:59.733889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:59.733979 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:59.734090 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:59.735521 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.736199 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.737629 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.738051 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.738194 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:59.742128 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:59.742133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:59.742136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:59.742167 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:59.744770 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:59.745754 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:59.745761 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:59.745765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:59.745923 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:59.746446 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:59.746449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:59.746451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:59.746703 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:59.746814 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:59.747757 (+ 943us) server_negotiation.cc:299] Negotiation successful
0809 01:04:59.747923 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 01:04:59.749555 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.731060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:38459 (local address 127.0.0.1:59256)
0809 01:04:59.731893 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.731986 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:59.733618 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:59.734309 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:59.734317 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:59.734794 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:59.735359 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:59.735378 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.736378 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.736390 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:59.737475 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:59.737487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.738288 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.738295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:59.738519 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:59.739218 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:04:59.739239 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:59.741928 (+ 2689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:59.744911 (+ 2983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:59.744922 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:59.744939 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:59.745608 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:59.746045 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:59.746051 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:59.746057 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:59.746312 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:59.746841 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:59.746851 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:59.747330 (+ 479us) client_negotiation.cc:769] Sending connection context
0809 01:04:59.747663 (+ 333us) client_negotiation.cc:240] Negotiation successful
0809 01:04:59.748192 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":412,"threads_started":1}
WARNING: no policy specified for kudu/127.4.82.188@KRBTEST.COM; defaulting to no policy
I20260809 01:04:59.751217 28373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.188@KRBTEST.COM" created.
Aug 09 01:04:59 dist-test-slave-07c2 krb5kdc[28211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.189@KRBTEST.COM for kudu/127.4.82.189@KRBTEST.COM
I20260809 01:04:59.767922 28378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.752415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35564 (local address 127.4.82.189:42171)
0809 01:04:59.753105 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:59.753115 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:59.753147 (+ 32us) server_negotiation.cc:407] Connection header received
0809 01:04:59.753201 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:59.753211 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:59.753456 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:59.753568 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:59.754501 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.755574 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.757348 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.757831 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.757921 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:59.761456 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:59.761471 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:59.761489 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:59.761527 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:59.764984 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:59.765828 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:59.765834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:59.765836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:59.765948 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:59.766352 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:59.766355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:59.766357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:59.766656 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:59.766809 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:59.767332 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 01:04:59.767638 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":481,"thread_start_us":356,"threads_started":1}
I20260809 01:04:59.768013 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.752131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:42171 (local address 127.0.0.1:35564)
0809 01:04:59.752284 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.752305 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:59.752457 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:59.753621 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:59.753626 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:59.754099 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:59.754320 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:59.754334 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.755711 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.755719 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:59.757209 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:59.757227 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.757981 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.757989 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:59.758225 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:59.758787 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:04:59.758814 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:59.761228 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:59.765121 (+ 3893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:59.765129 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:59.765133 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:59.765716 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:59.766056 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:59.766064 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:59.766071 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:59.766242 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:59.766799 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:59.766807 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:59.766997 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 01:04:59.767170 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 01:04:59.767523 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 01:04:59.770586 28373 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:04:59.771775 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.771361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44486)
0809 01:04:59.771518 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.771650 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 01:04:59.772138 28322 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
I20260809 01:04:59.773697 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.773405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44498)
0809 01:04:59.773534 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.773607 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 01:04:59.773952 28373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:42243: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.188.keytab.
Entry for principal kudu/127.4.82.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.188.keytab.
I20260809 01:04:59.788213 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.188:42243
--webserver_interface=127.4.82.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.188.keytab
--principal=kudu/127.4.82.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:38459,127.4.82.189:42171,127.4.82.188:42243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:04:59.823241 28373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } attempt: 1
I20260809 01:04:59.824388 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.823961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44514)
0809 01:04:59.824143 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.824244 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 01:04:59.825325 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.824977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44518)
0809 01:04:59.825127 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.825228 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 01:04:59.825523 28373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:42243: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
I20260809 01:04:59.893821 28373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } attempt: 2
I20260809 01:04:59.894909 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.894452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44532)
0809 01:04:59.894659 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.894759 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 01:04:59.895982 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.895657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44548)
0809 01:04:59.895801 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.895873 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 01:04:59.896194 28373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:42243: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
I20260809 01:04:59.905865 28296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } attempt: 5
Aug 09 01:04:59 dist-test-slave-07c2 krb5kdc[28211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for kudu/127.4.82.189@KRBTEST.COM
I20260809 01:04:59.920166 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.906506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:42171 (local address 127.0.0.1:35570)
0809 01:04:59.906733 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.906754 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:04:59.906875 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:04:59.907206 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:04:59.907210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:04:59.907626 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:04:59.907834 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:59.907844 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.909261 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.909265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:59.910202 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:04:59.910215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.911379 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.911384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:04:59.911554 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:59.912201 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:04:59.912218 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:04:59.914187 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:04:59.917812 (+ 3625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:59.917816 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:59.917819 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:59.918172 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:59.918613 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:04:59.918617 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:04:59.918619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:04:59.918750 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:04:59.919411 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:04:59.919414 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:04:59.919547 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 01:04:59.919659 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 01:04:59.919854 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 01:04:59.920394 28378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.906693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35570 (local address 127.4.82.189:42171)
0809 01:04:59.906869 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 01:04:59.906875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:04:59.906896 (+ 21us) server_negotiation.cc:407] Connection header received
0809 01:04:59.906991 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:04:59.906996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:04:59.907071 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:04:59.907173 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:04:59.907990 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.909115 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.910373 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:04:59.911162 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:04:59.911331 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:04:59.914381 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:04:59.914387 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:04:59.914392 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:04:59.914428 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:04:59.917678 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:59.918304 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:59.918313 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:59.918318 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:59.918489 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:04:59.918870 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:04:59.918878 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:04:59.918881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:04:59.919261 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:04:59.919435 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 01:04:59.919801 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 01:04:59.920013 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 01:04:59.921567 28296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:04:59.922693 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.922327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44564)
0809 01:04:59.922488 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.922557 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 01:04:59.923625 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.923328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44568)
0809 01:04:59.923462 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.923545 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 01:04:59.923866 28296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:42243: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
I20260809 01:04:59.975124 28296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } attempt: 1
I20260809 01:04:59.976267 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.975799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44574)
0809 01:04:59.975997 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.976091 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 01:04:59.977355 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.976994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44580)
0809 01:04:59.977175 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.977250 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260809 01:04:59.977694 28296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:42243: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
I20260809 01:04:59.987370 28373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } attempt: 3
I20260809 01:04:59.988300 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.987894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44588)
0809 01:04:59.988078 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.988169 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 01:04:59.989364 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:04:59.988973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44594)
0809 01:04:59.989197 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 01:04:59.989264 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260809 01:04:59.989625 28373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:42243: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
I20260809 01:05:00.045956 28296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } attempt: 2
I20260809 01:05:00.047219 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.046741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44598)
0809 01:05:00.046950 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:00.047052 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 01:05:00.048240 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.047943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44612)
0809 01:05:00.048065 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:00.048125 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 01:05:00.048532 28296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:42243: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
W20260809 01:05:00.119669 28381 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 01:05:00.119997 28381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:05:00.120141 28381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:05:00.120265 28381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:05:00.127252 28381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:05:00.127411 28381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:05:00.127521 28381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:05:00.127609 28381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:05:00.127697 28381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:05:00.132830 28373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } attempt: 4
I20260809 01:05:00.133919 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.133446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44628)
0809 01:05:00.133684 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:00.133769 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 01:05:00.134886 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.134577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44644)
0809 01:05:00.134724 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:00.134791 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 01:05:00.135128 28373 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:42243: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
I20260809 01:05:00.137552 28381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:38459,127.4.82.189:42171,127.4.82.188:42243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.188.keytab
--principal=kudu/127.4.82.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.188:42243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.82.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:05:00.138758 28381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:05:00.140036 28381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:05:00.151505 28389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:05:00.151594 28387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:05:00.152011 28386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:05:00.152237 28381 server_base.cc:1034] running on GCE node
I20260809 01:05:00.152941 28381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:05:00.154292 28381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:05:00.154764 28296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } attempt: 3
I20260809 01:05:00.155550 28381 hybrid_clock.cc:648] HybridClock initialized: now 1786237500155482 us; error 49 us; skew 500 ppm
I20260809 01:05:00.155843 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.155387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44656)
0809 01:05:00.155573 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:00.155650 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 01:05:00.155900 28381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 01:05:00.156817 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.156420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44658)
0809 01:05:00.156618 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:00.156704 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260809 01:05:00.157088 28296 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:42243: Network error: Client connection negotiation failed: client connection to 127.4.82.188:42243: connect: Connection refused (error 111)
Aug 09 01:05:00 dist-test-slave-07c2 krb5kdc[28211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:05:00.159106 28381 init.cc:377] Logged in from keytab as kudu/127.4.82.188@KRBTEST.COM (short username kudu)
I20260809 01:05:00.160451 28381 webserver.cc:460] Webserver started at http://127.4.82.188:39487/ using document root <none> and password file <none>
I20260809 01:05:00.161079 28381 fs_manager.cc:359] Metadata directory not provided
I20260809 01:05:00.161198 28381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:05:00.161514 28381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:05:00.164527 28381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/data/instance:
uuid: "b9143a88e1f548ae9b98b91797cf86c5"
format_stamp: "Formatted at 2026-08-09 01:05:00 on dist-test-slave-07c2"
server_key: "b2ff7ca5847f0e902e8ba3ac7a7d8642"
server_key_iv: "3ccc3729e265b9b5261c4e2ff10e76a2"
server_key_version: "encryptionkey@0"
I20260809 01:05:00.165314 28381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/wal/instance:
uuid: "b9143a88e1f548ae9b98b91797cf86c5"
format_stamp: "Formatted at 2026-08-09 01:05:00 on dist-test-slave-07c2"
server_key: "b2ff7ca5847f0e902e8ba3ac7a7d8642"
server_key_iv: "3ccc3729e265b9b5261c4e2ff10e76a2"
server_key_version: "encryptionkey@0"
I20260809 01:05:00.170126 28381 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 01:05:00.173377 28396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:05:00.174618 28381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 01:05:00.174789 28381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/wal
uuid: "b9143a88e1f548ae9b98b91797cf86c5"
format_stamp: "Formatted at 2026-08-09 01:05:00 on dist-test-slave-07c2"
server_key: "b2ff7ca5847f0e902e8ba3ac7a7d8642"
server_key_iv: "3ccc3729e265b9b5261c4e2ff10e76a2"
server_key_version: "encryptionkey@0"
I20260809 01:05:00.174999 28381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:05:00.213120 28381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:05:00.217130 28381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:05:00.217406 28381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:05:00.236243 28381 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.188:42243
I20260809 01:05:00.236275 28451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.188:42243 every 8 connection(s)
I20260809 01:05:00.237597 28381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/data/info.pb
I20260809 01:05:00.241189 28452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:05:00.245292 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 28381
I20260809 01:05:00.245484 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786237144174446-4426-0/minicluster-data/master-2/wal/instance
I20260809 01:05:00.246055 4426 external_mini_cluster.cc:1428] Setting key 98d5568fae5524ba04a189865057ac68
I20260809 01:05:00.248683 28452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:05:00 dist-test-slave-07c2 krb5kdc[28211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.188@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
Aug 09 01:05:00 dist-test-slave-07c2 krb5kdc[28211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.188@KRBTEST.COM
I20260809 01:05:00.268812 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.252395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59260 (local address 127.4.82.190:38459)
0809 01:05:00.252925 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.252933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.253452 (+ 519us) server_negotiation.cc:407] Connection header received
0809 01:05:00.253652 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.253660 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.253745 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.253859 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.255013 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.255839 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.257393 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.257949 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.258072 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.261767 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:00.261775 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:00.261782 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:00.261827 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:00.265235 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.266072 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.266079 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.266083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.266257 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.266748 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.266753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.266756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.267119 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:00.267253 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:00.268068 (+ 815us) server_negotiation.cc:299] Negotiation successful
0809 01:05:00.268324 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":274,"threads_started":1}
I20260809 01:05:00.269882 28458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.252377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:38459 (local address 127.0.0.1:59260)
0809 01:05:00.253291 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:00.253340 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:00.253504 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:00.253910 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:00.253914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:00.254503 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:00.254836 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:00.254848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.255993 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.255999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:00.257233 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:00.257249 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.258186 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.258194 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:00.258390 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.258981 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:00.259010 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:00.261571 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:00.265368 (+ 3797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:00.265379 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:00.265391 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:00.265956 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:00.266404 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:00.266409 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:00.266412 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:00.266614 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:00.267253 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:00.267260 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:00.267685 (+ 425us) client_negotiation.cc:769] Sending connection context
0809 01:05:00.267975 (+ 290us) client_negotiation.cc:240] Negotiation successful
0809 01:05:00.268493 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":5,"thread_start_us":550,"threads_started":1}
I20260809 01:05:00.269903 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.248052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44672 (local address 127.4.82.188:42243)
0809 01:05:00.248921 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.248932 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.248972 (+ 40us) server_negotiation.cc:407] Connection header received
0809 01:05:00.250620 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.250641 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.251064 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.251447 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.253230 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.254854 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.256400 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.257027 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.257168 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.262546 (+ 5378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:00.262553 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:00.262563 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:00.262590 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:00.265664 (+ 3074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.266415 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.266423 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.266425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.266614 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.267019 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.267022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.267024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.267709 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:00.268033 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:00.268346 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 01:05:00.268603 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":636,"spinlock_wait_cycles":17152,"thread_start_us":399,"threads_started":1}
I20260809 01:05:00.271519 28452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:05:00 dist-test-slave-07c2 krb5kdc[28211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.188@KRBTEST.COM for kudu/127.4.82.189@KRBTEST.COM
Aug 09 01:05:00 dist-test-slave-07c2 krb5kdc[28211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:05:00.286566 28458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.272352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:42171 (local address 127.0.0.1:35584)
0809 01:05:00.272614 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:00.272633 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:00.272744 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:00.273096 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:00.273100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:00.273568 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:00.273752 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:00.273764 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.275133 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.275138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:00.276485 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:00.276496 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.277392 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.277396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:00.277691 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.278234 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:00.278256 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:00.281005 (+ 2749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:00.284236 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:00.284244 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:00.284251 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:00.284779 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:00.285156 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:00.285161 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:00.285167 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:00.285335 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:00.285831 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:00.285836 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:00.286012 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 01:05:00.286160 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 01:05:00.286343 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260809 01:05:00.287091 28378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.272306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35584 (local address 127.4.82.189:42171)
0809 01:05:00.272519 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.272523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.272702 (+ 179us) server_negotiation.cc:407] Connection header received
0809 01:05:00.272849 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.272852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.272951 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.273051 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.273908 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.275000 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.276610 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.277242 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.277363 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.281186 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:00.281191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:00.281194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:00.281225 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:00.284106 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.284901 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.284905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.284907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.285038 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.285445 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.285448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.285450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.285711 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:00.285819 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:00.286300 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 01:05:00.286521 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 01:05:00.288120 28452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:05:00.288951 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.273216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59270 (local address 127.4.82.190:38459)
0809 01:05:00.273380 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.273387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.273406 (+ 19us) server_negotiation.cc:407] Connection header received
0809 01:05:00.273486 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.273490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.273574 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.273701 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.275125 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.276227 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.277771 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.278390 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.278528 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.282951 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:00.282956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:00.282959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:00.282987 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:00.285936 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.286768 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.286776 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.286783 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.286925 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.287390 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.287396 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.287399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.287728 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:00.287998 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:00.288372 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 01:05:00.288609 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 01:05:00 dist-test-slave-07c2 krb5kdc[28211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.188@KRBTEST.COM for kudu/127.4.82.188@KRBTEST.COM
Aug 09 01:05:00 dist-test-slave-07c2 krb5kdc[28211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.189@KRBTEST.COM
I20260809 01:05:00.302394 28458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.288899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44682)
0809 01:05:00.289050 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:00.289069 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:00.289156 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:00.289713 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:00.289721 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:00.290161 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:00.290341 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:00.290350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.291662 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.291666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:00.292638 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:00.292655 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.293337 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.293345 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:00.293531 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.294104 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:00.294120 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:00.296395 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:00.299958 (+ 3563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:00.299966 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:00.299970 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:00.300486 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:00.300902 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:00.300906 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:00.300908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:00.301030 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:00.301638 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:00.301641 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:00.301770 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 01:05:00.301897 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 01:05:00.302091 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 01:05:00.302621 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.289243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44682 (local address 127.4.82.188:42243)
0809 01:05:00.289406 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.289413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.289433 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:05:00.289480 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.289486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.289557 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.289655 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.290470 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.291526 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.292784 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.293186 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.293279 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.296587 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:00.296594 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:00.296599 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:00.296638 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:00.299832 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.300615 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.300621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.300624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.300783 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.301133 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.301139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.301142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.301512 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:00.301669 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:00.302020 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 01:05:00.302266 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 01:05:00.306378 28378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.290550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35600 (local address 127.4.82.189:42171)
0809 01:05:00.290701 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.290705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.290779 (+ 74us) server_negotiation.cc:407] Connection header received
0809 01:05:00.290963 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.290967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.291029 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.291114 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.292446 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.293271 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.294653 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.295423 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.295521 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.300719 (+ 5198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:00.300725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:00.300730 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:00.300767 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:00.303729 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.304555 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.304558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.304560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.304684 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.305099 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.305104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.305106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.305370 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:00.305575 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:00.305915 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 01:05:00.306076 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 01:05:00.308423 28452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5: Bootstrap starting.
I20260809 01:05:00.311040 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:05:00.312187 28452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5: Neither blocks nor log segments found. Creating new log.
I20260809 01:05:00.313433 28452 log.cc:824] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5: Log is configured to *not* fsync() on all Append() calls
I20260809 01:05:00.316131 28452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5: No bootstrap required, opened a new log
I20260809 01:05:00.314568 28270 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7ba1371b42594538bb0c15fff07acdd4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59270
I20260809 01:05:00.322860 28452 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } }
I20260809 01:05:00.323186 28452 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9143a88e1f548ae9b98b91797cf86c5, State: Initialized, Role: FOLLOWER
I20260809 01:05:00.323627 28452 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } }
I20260809 01:05:00.324597 28460 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } } }
I20260809 01:05:00.324978 28460 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:00.325434 28452 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:05:00.334297 28296 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } attempt: 4
W20260809 01:05:00.334666 28471 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 01:05:00.334805 28471 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 01:05:00 dist-test-slave-07c2 krb5kdc[28211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for kudu/127.4.82.188@KRBTEST.COM
I20260809 01:05:00.332587 28381 master_runner.cc:186] Error getting master registration for 127.4.82.190:38459: OK, instance_id { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" instance_seqno: 1786237499193427 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 01:05:00.344478 28381 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:05:00.350353 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.335474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44688 (local address 127.4.82.188:42243)
0809 01:05:00.335655 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.335669 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.335689 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:05:00.335746 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.335757 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.335828 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.335938 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.336697 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.337918 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.339109 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.339873 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.340017 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.343932 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:00.343940 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:00.343943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:00.343975 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:00.347702 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.348322 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.348330 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.348334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.348459 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.348878 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.348883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.348887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.349218 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:00.349374 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:00.349748 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 01:05:00.349952 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 01:05:00.350345 28299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.334869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44688)
0809 01:05:00.335031 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:00.335050 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:00.335201 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:00.335964 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:00.335970 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:00.336383 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:00.336540 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:00.336548 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.338060 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.338067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:00.338963 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:00.338974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.340031 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.340037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:00.340278 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.340957 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:00.340981 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:00.343730 (+ 2749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:00.347826 (+ 4096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:00.347831 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:00.347835 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:00.348202 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:00.348590 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:00.348596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:00.348599 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:00.348759 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:00.349352 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:00.349357 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:00.349505 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 01:05:00.349648 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 01:05:00.349947 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 01:05:00.354926 28296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4: Bootstrap starting.
I20260809 01:05:00.357997 28296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4: Neither blocks nor log segments found. Creating new log.
I20260809 01:05:00.359040 28296 log.cc:824] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4: Log is configured to *not* fsync() on all Append() calls
I20260809 01:05:00.361779 28296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4: No bootstrap required, opened a new log
I20260809 01:05:00.367486 28296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } }
I20260809 01:05:00.367812 28296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba1371b42594538bb0c15fff07acdd4, State: Initialized, Role: FOLLOWER
I20260809 01:05:00.368261 28296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } }
I20260809 01:05:00.369225 28474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } } }
I20260809 01:05:00.369626 28474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:00.370340 28296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:05:00.370523 28270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 01:05:00.370790 28270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:05:00.374330 28270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } }
I20260809 01:05:00.378357 28270 leader_election.cc:288] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [CANDIDATE]: Term 1 election: Requested vote from peers c5a7573f0aad4d7b837e25a3491b49cd (127.4.82.189:42171), b9143a88e1f548ae9b98b91797cf86c5 (127.4.82.188:42243)
I20260809 01:05:00.378475 28424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ba1371b42594538bb0c15fff07acdd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b9143a88e1f548ae9b98b91797cf86c5"
I20260809 01:05:00.378999 28424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:05:00.383160 28424 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba1371b42594538bb0c15fff07acdd4 in term 1.
I20260809 01:05:00.384250 28244 leader_election.cc:302] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba1371b42594538bb0c15fff07acdd4, b9143a88e1f548ae9b98b91797cf86c5; no voters:
I20260809 01:05:00.384912 28474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:05:00.395277 28474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [term 1 LEADER]: Becoming Leader. State: Replica: 7ba1371b42594538bb0c15fff07acdd4, State: Running, Role: LEADER
I20260809 01:05:00.396199 28474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } }
I20260809 01:05:00.407445 28487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ba1371b42594538bb0c15fff07acdd4. Latest consensus state: current_term: 1 leader_uuid: "7ba1371b42594538bb0c15fff07acdd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } } }
I20260809 01:05:00.407864 28492 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 01:05:00.407925 28487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [sys.catalog]: This master's current role is: LEADER
I20260809 01:05:00.412887 28224 master_runner.cc:186] Error getting master registration for 127.4.82.189:42171: OK, instance_id { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" instance_seqno: 1786237499707861 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260809 01:05:00.432976 28247 tablet.cc:2369] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:05:00.433952 28224 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:05:00.434521 28373 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } attempt: 5
I20260809 01:05:00.433627 28493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:05:00.458681 28493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:05:00.459672 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.413376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59276 (local address 127.4.82.190:38459)
0809 01:05:00.413577 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.413586 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.427835 (+ 14249us) server_negotiation.cc:407] Connection header received
0809 01:05:00.428142 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.428149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.428239 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.428323 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.434195 (+ 5872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.435453 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.439799 (+ 4346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.440450 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.440548 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.444216 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:00.444222 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:00.444227 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:00.444269 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:00.447932 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.449455 (+ 1523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.449462 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.449465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.449619 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.450437 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.450445 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.450449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.450860 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:00.451156 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:00.458998 (+ 7842us) server_negotiation.cc:299] Negotiation successful
0809 01:05:00.459269 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 01:05:00.463521 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.419634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44694 (local address 127.4.82.188:42243)
0809 01:05:00.419829 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.419834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.432935 (+ 13101us) server_negotiation.cc:407] Connection header received
0809 01:05:00.433557 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.433563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.433708 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.433789 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.439091 (+ 5302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.440235 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.445650 (+ 5415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.446467 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.446576 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.451893 (+ 5317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:00.451899 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:00.451907 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:00.451952 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:00.455929 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.458588 (+ 2659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.458597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.458601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.458779 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.459244 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.459249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.459251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.459603 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:00.459863 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:00.462196 (+ 2333us) server_negotiation.cc:299] Negotiation successful
0809 01:05:00.462762 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 01:05:00.466809 28378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.427989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35608 (local address 127.4.82.189:42171)
0809 01:05:00.428168 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.428174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.436460 (+ 8286us) server_negotiation.cc:407] Connection header received
0809 01:05:00.436697 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.436702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.436770 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.436835 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.442435 (+ 5600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.443619 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.448348 (+ 4729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.449174 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.449319 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.456085 (+ 6766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:00.456092 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:00.456096 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:00.456147 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:00.459800 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.462485 (+ 2685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.462492 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.462497 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.462714 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.463359 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.463364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.463368 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.463782 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:00.465609 (+ 1827us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:00.466170 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 01:05:00.466430 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 01:05:00.469403 28402 tablet.cc:2369] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 01:05:00 dist-test-slave-07c2 krb5kdc[28211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.189@KRBTEST.COM for kudu/127.4.82.188@KRBTEST.COM
I20260809 01:05:00.483884 28424 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5 [term 1 FOLLOWER]: Refusing update from remote peer 7ba1371b42594538bb0c15fff07acdd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:05:00.484892 28487 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:05:00.492110 28497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.435714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44708 (local address 127.4.82.188:42243)
0809 01:05:00.451934 (+ 16220us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.451939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.451960 (+ 21us) server_negotiation.cc:407] Connection header received
0809 01:05:00.452006 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.452012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.452121 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.452237 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.462296 (+ 10059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.464426 (+ 2130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.467089 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.467786 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.476310 (+ 8524us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.482422 (+ 6112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:00.482429 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:00.482434 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:00.482474 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:00.486479 (+ 4005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.488889 (+ 2410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.488895 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.488899 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.489082 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:00.489711 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:00.489716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:00.489719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:00.490100 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:00.490242 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:00.491473 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0809 01:05:00.491690 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16076,"thread_start_us":15832,"threads_started":1}
I20260809 01:05:00.492786 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:00.435032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:42243 (local address 127.0.0.1:44708)
0809 01:05:00.435394 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:00.435412 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:00.435516 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:00.452281 (+ 16765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:00.452287 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:00.452776 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:00.452971 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:00.452982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.464572 (+ 11590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.464578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:00.466050 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:00.466069 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:00.475447 (+ 9378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:00.475453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:00.475691 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:00.476468 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:00.476491 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:00.482146 (+ 5655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:00.488144 (+ 5998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:00.488149 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:00.488154 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:00.488712 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:00.489403 (+ 691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:00.489409 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:00.489413 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:00.489576 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:00.490320 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:00.490326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:00.490489 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 01:05:00.490631 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 01:05:00.490924 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230}
I20260809 01:05:00.498836 28373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd: Bootstrap starting.
I20260809 01:05:00.500512 28460 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ba1371b42594538bb0c15fff07acdd4. Latest consensus state: current_term: 1 leader_uuid: "7ba1371b42594538bb0c15fff07acdd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } } }
I20260809 01:05:00.500792 28460 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:00.502753 28373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd: Neither blocks nor log segments found. Creating new log.
I20260809 01:05:00.504262 28500 mvcc.cc:205] Tried to move back new op lower bound from 7316428801969868800 to 7316428801630777344. Current Snapshot: MvccSnapshot[applied={T|T < 7316428801969868800}]
I20260809 01:05:00.504577 28460 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ba1371b42594538bb0c15fff07acdd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } } }
I20260809 01:05:00.504809 28460 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:00.514197 28499 mvcc.cc:205] Tried to move back new op lower bound from 7316428801969868800 to 7316428801630777344. Current Snapshot: MvccSnapshot[applied={T|T < 7316428801969868800}]
I20260809 01:05:00.514571 28487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ba1371b42594538bb0c15fff07acdd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } } }
I20260809 01:05:00.514886 28487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [sys.catalog]: This master's current role is: LEADER
I20260809 01:05:00.517277 28373 log.cc:824] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd: Log is configured to *not* fsync() on all Append() calls
I20260809 01:05:00.520438 28493 catalog_manager.cc:1347] Generated new cluster ID: 3053662d812b450198f86bee12408b1a
I20260809 01:05:00.520529 28493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:05:00.521251 28373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd: No bootstrap required, opened a new log
I20260809 01:05:00.536566 28373 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } }
I20260809 01:05:00.537052 28373 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a7573f0aad4d7b837e25a3491b49cd, State: Initialized, Role: FOLLOWER
I20260809 01:05:00.537592 28373 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } }
I20260809 01:05:00.553578 28373 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:05:00.553978 28346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:05:00.555418 28347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ba1371b42594538bb0c15fff07acdd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c5a7573f0aad4d7b837e25a3491b49cd"
I20260809 01:05:00.559957 28346 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd [term 1 FOLLOWER]: Refusing update from remote peer 7ba1371b42594538bb0c15fff07acdd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:05:00.563411 28347 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba1371b42594538bb0c15fff07acdd4 in term 1.
I20260809 01:05:00.570816 28504 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:05:00.553267 28507 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } } }
I20260809 01:05:00.577732 28507 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:00.592592 28493 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:05:00.595213 28493 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:05:00.607133 28510 mvcc.cc:205] Tried to move back new op lower bound from 7316428801969868800 to 7316428801630777344. Current Snapshot: MvccSnapshot[applied={T|T < 7316428801969868800}]
I20260809 01:05:00.611269 28493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4: Generated new TSK 0
I20260809 01:05:00.612632 28493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:05:00.613340 28509 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ba1371b42594538bb0c15fff07acdd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } } }
I20260809 01:05:00.613780 28509 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:00.614307 28507 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ba1371b42594538bb0c15fff07acdd4. Latest consensus state: current_term: 1 leader_uuid: "7ba1371b42594538bb0c15fff07acdd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } } }
I20260809 01:05:00.614758 28507 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:00.616472 28487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ba1371b42594538bb0c15fff07acdd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1371b42594538bb0c15fff07acdd4" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38459 } } peers { permanent_uuid: "c5a7573f0aad4d7b837e25a3491b49cd" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 42171 } } peers { permanent_uuid: "b9143a88e1f548ae9b98b91797cf86c5" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 42243 } } }
I20260809 01:05:00.622828 28487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba1371b42594538bb0c15fff07acdd4 [sys.catalog]: This master's current role is: LEADER
I20260809 01:05:00.667109 28525 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 01:05:00.783155 28457 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:00.773858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59280 (local address 127.4.82.190:38459)
0809 01:05:00.774073 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.774080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.781095 (+ 7015us) server_negotiation.cc:407] Connection header received
0809 01:05:00.781304 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.781310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.781403 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.781509 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.782896 (+ 1387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59280: BlockingRecv error: recv got EOF from 127.0.0.1:59280 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 01:05:00.785059 28497 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:00.780142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44722 (local address 127.4.82.188:42243)
0809 01:05:00.780500 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.780507 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.783544 (+ 3037us) server_negotiation.cc:407] Connection header received
0809 01:05:00.783612 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.783618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.783710 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.783809 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.784843 (+ 1034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44722: BlockingRecv error: recv got EOF from 127.0.0.1:44722 (error 108)
Metrics: {"server-negotiator.queue_time_us":233}
W20260809 01:05:00.786769 28378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:00.780856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35616 (local address 127.4.82.189:42171)
0809 01:05:00.781046 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.781052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.785474 (+ 4422us) server_negotiation.cc:407] Connection header received
0809 01:05:00.785558 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.785563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.785642 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.785749 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.786554 (+ 805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35616: BlockingRecv error: recv got EOF from 127.0.0.1:35616 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 01:05:00.807474 28378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:00.805010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.4.82.189:42171)
0809 01:05:00.805193 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.805200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.805695 (+ 495us) server_negotiation.cc:407] Connection header received
0809 01:05:00.805908 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.805915 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.806061 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.806199 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.807146 (+ 947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35620: BlockingRecv error: recv got EOF from 127.0.0.1:35620 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 01:05:00.817040 28378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:00.812004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35636 (local address 127.4.82.189:42171)
0809 01:05:00.812167 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.812173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.815516 (+ 3343us) server_negotiation.cc:407] Connection header received
0809 01:05:00.815731 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.815737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.815818 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.815923 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.816834 (+ 911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35636: BlockingRecv error: recv got EOF from 127.0.0.1:35636 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 01:05:00.824512 28378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:00.822386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.4.82.189:42171)
0809 01:05:00.822568 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.822582 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.823060 (+ 478us) server_negotiation.cc:407] Connection header received
0809 01:05:00.823268 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.823274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.823379 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.823471 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.824287 (+ 816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35646: BlockingRecv error: recv got EOF from 127.0.0.1:35646 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 01:05:00.834923 28378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:00.832514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35656 (local address 127.4.82.189:42171)
0809 01:05:00.832748 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.832754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.833300 (+ 546us) server_negotiation.cc:407] Connection header received
0809 01:05:00.833553 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.833559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.833667 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.833754 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.834722 (+ 968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35656: BlockingRecv error: recv got EOF from 127.0.0.1:35656 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
W20260809 01:05:00.849701 28378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:00.847380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35672 (local address 127.4.82.189:42171)
0809 01:05:00.847590 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.847598 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.848196 (+ 598us) server_negotiation.cc:407] Connection header received
0809 01:05:00.848397 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.848402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.848487 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.848605 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.849483 (+ 878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35672: BlockingRecv error: recv got EOF from 127.0.0.1:35672 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 01:05:00.872426 28378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:00.869711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.4.82.189:42171)
0809 01:05:00.869924 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.869931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.870516 (+ 585us) server_negotiation.cc:407] Connection header received
0809 01:05:00.870893 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.870899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.870997 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.871097 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.872149 (+ 1052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35674: BlockingRecv error: recv got EOF from 127.0.0.1:35674 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 01:05:00.912788 28378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:00.910410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35688 (local address 127.4.82.189:42171)
0809 01:05:00.910597 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.910604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.911159 (+ 555us) server_negotiation.cc:407] Connection header received
0809 01:05:00.911366 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.911373 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.911477 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.911596 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.912510 (+ 914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35688: BlockingRecv error: recv got EOF from 127.0.0.1:35688 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 01:05:00.982283 28378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:00.980352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35700 (local address 127.4.82.189:42171)
0809 01:05:00.980564 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:00.980570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:00.980937 (+ 367us) server_negotiation.cc:407] Connection header received
0809 01:05:00.981105 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:00.981111 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:00.981210 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:00.981329 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:00.982059 (+ 730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35700: BlockingRecv error: recv got EOF from 127.0.0.1:35700 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 01:05:01.115898 28378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:01.114079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35716 (local address 127.4.82.189:42171)
0809 01:05:01.114308 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:01.114312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:01.114585 (+ 273us) server_negotiation.cc:407] Connection header received
0809 01:05:01.114755 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:01.114759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:01.114826 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:01.114912 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:01.115718 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35716: BlockingRecv error: recv got EOF from 127.0.0.1:35716 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 01:05:01.336753 28471 catalog_manager.cc:1259] Loaded cluster ID: 3053662d812b450198f86bee12408b1a
I20260809 01:05:01.336892 28471 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5: loading cluster ID for follower catalog manager: success
I20260809 01:05:01.339351 28471 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5: acquiring CA information for follower catalog manager: success
I20260809 01:05:01.341068 28471 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b9143a88e1f548ae9b98b91797cf86c5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 01:05:01.377197 28378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:01.375423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35726 (local address 127.4.82.189:42171)
0809 01:05:01.375597 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:01.375602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:01.375906 (+ 304us) server_negotiation.cc:407] Connection header received
0809 01:05:01.376088 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:01.376092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:01.376165 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:01.376266 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:01.377036 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35726: BlockingRecv error: recv got EOF from 127.0.0.1:35726 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 01:05:01.671531 28525 catalog_manager.cc:1259] Loaded cluster ID: 3053662d812b450198f86bee12408b1a
I20260809 01:05:01.671698 28525 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd: loading cluster ID for follower catalog manager: success
I20260809 01:05:01.674180 28525 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd: acquiring CA information for follower catalog manager: success
I20260809 01:05:01.676100 28525 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c5a7573f0aad4d7b837e25a3491b49cd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 01:05:01.900251 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:01.893136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35738 (local address 127.4.82.189:42171)
0809 01:05:01.893989 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:01.893994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:01.894011 (+ 17us) server_negotiation.cc:407] Connection header received
0809 01:05:01.894073 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:01.894077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:01.894226 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:01.894327 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:01.894885 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:01.895840 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:01.897843 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:01.898258 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:01.898350 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:01.898930 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:01.899539 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:01.899646 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:01.899817 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 01:05:01.899877 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":750,"thread_start_us":565,"threads_started":1}
I20260809 01:05:01.913465 28600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:01.907173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44728 (local address 127.4.82.188:42243)
0809 01:05:01.907868 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:01.907873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:01.907888 (+ 15us) server_negotiation.cc:407] Connection header received
0809 01:05:01.907969 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:01.907973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:01.908107 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:01.908247 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:01.908712 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:01.909406 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:01.911244 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:01.911665 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:01.911754 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:01.912344 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:01.912809 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:01.912918 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:01.913075 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 01:05:01.913134 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":454,"threads_started":1}
I20260809 01:05:01.941890 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:01.921244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35742 (local address 127.4.82.189:42171)
0809 01:05:01.921471 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:01.921480 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:01.934444 (+ 12964us) server_negotiation.cc:407] Connection header received
0809 01:05:01.934681 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:01.934689 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:01.934773 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:01.934879 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:01.935712 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:01.936840 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:01.939489 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:01.940207 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:01.940331 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:01.940974 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:01.941106 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:01.941187 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:01.941423 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 01:05:01.941587 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 01:05:01.942929 28608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:01.921662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59292 (local address 127.4.82.190:38459)
0809 01:05:01.933008 (+ 11346us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:01.933017 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:01.933042 (+ 25us) server_negotiation.cc:407] Connection header received
0809 01:05:01.933182 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:01.933190 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:01.933447 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:01.933635 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:01.935167 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:01.936658 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:01.940158 (+ 3500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:01.940823 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:01.940940 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:01.941555 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:01.941982 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:01.942086 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:01.942551 (+ 465us) server_negotiation.cc:299] Negotiation successful
0809 01:05:01.942635 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11184,"thread_start_us":11037,"threads_started":1}
I20260809 01:05:01.946450 28600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:01.921920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44738 (local address 127.4.82.188:42243)
0809 01:05:01.922074 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:01.922080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:01.939596 (+ 17516us) server_negotiation.cc:407] Connection header received
0809 01:05:01.939725 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:01.939730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:01.939812 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:01.939903 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:01.940726 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:01.941826 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:01.944367 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:01.945020 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:01.945114 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:01.945746 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:01.945867 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:01.945974 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:01.946111 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 01:05:01.946197 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 01:05:01.947391 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 28224
I20260809 01:05:01.948354 28291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:05:02.059415 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 28224
I20260809 01:05:02.083374 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 28303
I20260809 01:05:02.084481 28368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:05:02.189841 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 28303
I20260809 01:05:02.212214 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 28381
I20260809 01:05:02.213392 28446 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:05:02.318604 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 28381
2026-08-09T01:05:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4976 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 01:05:02 dist-test-slave-07c2 krb5kdc[28614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 01:05:02 dist-test-slave-07c2 krb5kdc[28614](info): set up 2 sockets
Aug 09 01:05:02 dist-test-slave-07c2 krb5kdc[28614](info): commencing operation
krb5kdc: starting...
W20260809 01:05:03.471258 4426 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/test-user.keytab.
Aug 09 01:05:03 dist-test-slave-07c2 krb5kdc[28614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T01:05:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T01:05:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.82.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
Entry for principal kudu/127.4.82.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab.
I20260809 01:05:03.781736 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:38677
--webserver_interface=127.4.82.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:45235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:38677,127.4.82.189:36625,127.4.82.188:36533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 01:05:04.110553 28628 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 01:05:04.110937 28628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:05:04.111068 28628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:05:04.111233 28628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:05:04.118249 28628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:05:04.118369 28628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:05:04.118470 28628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:05:04.118569 28628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:05:04.118669 28628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:05:04.128170 28628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:45235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:38677,127.4.82.189:36625,127.4.82.188:36533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.190.keytab
--principal=kudu/127.4.82.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.190:38677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.82.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:05:04.129446 28628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:05:04.130722 28628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:05:04.142177 28633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:05:04.142153 28636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:05:04.142203 28634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:05:04.143105 28628 server_base.cc:1034] running on GCE node
I20260809 01:05:04.143913 28628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:05:04.145234 28628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:05:04.146450 28628 hybrid_clock.cc:648] HybridClock initialized: now 1786237504146396 us; error 28 us; skew 500 ppm
I20260809 01:05:04.146760 28628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:05:04 dist-test-slave-07c2 krb5kdc[28614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237504, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:05:04.149938 28628 init.cc:377] Logged in from keytab as kudu/127.4.82.190@KRBTEST.COM (short username kudu)
I20260809 01:05:04.151407 28628 webserver.cc:460] Webserver started at http://127.4.82.190:43523/ using document root <none> and password file <none>
I20260809 01:05:04.152103 28628 fs_manager.cc:359] Metadata directory not provided
I20260809 01:05:04.152235 28628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:05:04.152551 28628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:05:04.155548 28628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/data/instance:
uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3"
format_stamp: "Formatted at 2026-08-09 01:05:04 on dist-test-slave-07c2"
server_key: "849b13bb7e5888051b6cf6be8cc81e62"
server_key_iv: "f2b1c8cb197cadb5b197dfcd36a744d1"
server_key_version: "encryptionkey@0"
I20260809 01:05:04.156334 28628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/wal/instance:
uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3"
format_stamp: "Formatted at 2026-08-09 01:05:04 on dist-test-slave-07c2"
server_key: "849b13bb7e5888051b6cf6be8cc81e62"
server_key_iv: "f2b1c8cb197cadb5b197dfcd36a744d1"
server_key_version: "encryptionkey@0"
I20260809 01:05:04.161274 28628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 01:05:04.164556 28644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:05:04.165881 28628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 01:05:04.166067 28628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/wal
uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3"
format_stamp: "Formatted at 2026-08-09 01:05:04 on dist-test-slave-07c2"
server_key: "849b13bb7e5888051b6cf6be8cc81e62"
server_key_iv: "f2b1c8cb197cadb5b197dfcd36a744d1"
server_key_version: "encryptionkey@0"
I20260809 01:05:04.166287 28628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:05:04.180640 28628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:05:04.184383 28628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:05:04.184669 28628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:05:04.203540 28628 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.190:38677
I20260809 01:05:04.203563 28698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.190:38677 every 8 connection(s)
I20260809 01:05:04.204909 28628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/data/info.pb
I20260809 01:05:04.208461 28699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:05:04.209139 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 28628
I20260809 01:05:04.209347 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-0/wal/instance
I20260809 01:05:04.209801 4426 external_mini_cluster.cc:1428] Setting key aeb139915472a22f3146dc94a6e23448
I20260809 01:05:04.214902 28699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:05:04 dist-test-slave-07c2 krb5kdc[28614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237504, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:05:04.238935 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.218014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:38677 (local address 127.0.0.1:43530)
0809 01:05:04.219212 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.219252 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:04.221383 (+ 2131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:04.222309 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:04.222314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:04.222928 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:04.223472 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:04.223499 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.225046 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.225054 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:04.226984 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:04.227001 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.227982 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.227993 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:04.228233 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:04.228957 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:04.228989 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:04.231751 (+ 2762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:04.234858 (+ 3107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:04.234872 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:04.234882 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:04.235264 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:04.235690 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:04.235696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:04.235700 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:04.235860 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:04.236576 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:04.236583 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:04.236799 (+ 216us) client_negotiation.cc:769] Sending connection context
0809 01:05:04.237075 (+ 276us) client_negotiation.cc:240] Negotiation successful
0809 01:05:04.237653 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1005,"mutex_wait_us":147,"thread_start_us":809,"threads_started":1}
I20260809 01:05:04.238978 28701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.218013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43530 (local address 127.4.82.190:38677)
0809 01:05:04.218912 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:04.218930 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:04.219678 (+ 748us) server_negotiation.cc:407] Connection header received
0809 01:05:04.221739 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:04.221748 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:04.222052 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:04.222272 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:04.223714 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.224884 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.227132 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.227705 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.227832 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:04.231972 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:04.231980 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:04.231990 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:04.232016 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:04.234697 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:04.235422 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:04.235434 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:04.235442 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:04.235577 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:04.235974 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:04.235985 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:04.235987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:04.236434 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:04.236650 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:04.237215 (+ 565us) server_negotiation.cc:299] Negotiation successful
0809 01:05:04.237501 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":437,"threads_started":1}
WARNING: no policy specified for kudu/127.4.82.189@KRBTEST.COM; defaulting to no policy
I20260809 01:05:04.241729 28699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:05:04.242849 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.242463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:36625 (local address 127.0.0.1:35336)
0809 01:05:04.242639 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.242742 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 01:05:04.243201 28646 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.189@KRBTEST.COM" created.
I20260809 01:05:04.244791 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.244445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:36625 (local address 127.0.0.1:35348)
0809 01:05:04.244595 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.244654 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 01:05:04.245004 28699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:36625: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.189.keytab.
Entry for principal kudu/127.4.82.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.189.keytab.
I20260809 01:05:04.276522 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.189:36625
--webserver_interface=127.4.82.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.189.keytab
--principal=kudu/127.4.82.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:45235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:38677,127.4.82.189:36625,127.4.82.188:36533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:05:04.294298 28699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } attempt: 1
I20260809 01:05:04.295593 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.295076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:36625 (local address 127.0.0.1:35358)
0809 01:05:04.295330 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.295456 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 01:05:04.296649 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.296261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:36625 (local address 127.0.0.1:35360)
0809 01:05:04.296429 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.296510 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260809 01:05:04.296845 28699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:36625: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
I20260809 01:05:04.365067 28699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } attempt: 2
I20260809 01:05:04.366173 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.365703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:36625 (local address 127.0.0.1:35364)
0809 01:05:04.365953 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.366034 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 01:05:04.367136 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.366775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:36625 (local address 127.0.0.1:35374)
0809 01:05:04.366942 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.367004 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 01:05:04.367328 28699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:36625: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
I20260809 01:05:04.458585 28699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } attempt: 3
I20260809 01:05:04.459925 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.459484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:36625 (local address 127.0.0.1:35380)
0809 01:05:04.459683 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.459782 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 01:05:04.460929 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.460558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:36625 (local address 127.0.0.1:35396)
0809 01:05:04.460724 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.460794 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 01:05:04.461120 28699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:36625: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
I20260809 01:05:04.604364 28699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } attempt: 4
I20260809 01:05:04.605568 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.605063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:36625 (local address 127.0.0.1:35404)
0809 01:05:04.605352 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.605441 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260809 01:05:04.606541 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.606195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:36625 (local address 127.0.0.1:35416)
0809 01:05:04.606348 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.606425 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 01:05:04.606748 28699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.189:36625: Network error: Client connection negotiation failed: client connection to 127.4.82.189:36625: connect: Connection refused (error 111)
W20260809 01:05:04.607014 28706 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 01:05:04.607386 28706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:05:04.607489 28706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:05:04.607627 28706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:05:04.614584 28706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:05:04.614657 28706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:05:04.614712 28706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:05:04.614754 28706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:05:04.614869 28706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:05:04.624234 28706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:45235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:38677,127.4.82.189:36625,127.4.82.188:36533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.189.keytab
--principal=kudu/127.4.82.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.189:36625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.82.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:05:04.625392 28706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:05:04.626622 28706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:05:04.638041 28712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:05:04.638083 28715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:05:04.638127 28713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:05:04.644241 28706 server_base.cc:1034] running on GCE node
I20260809 01:05:04.645009 28706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:05:04.646257 28706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:05:04.647465 28706 hybrid_clock.cc:648] HybridClock initialized: now 1786237504647392 us; error 47 us; skew 500 ppm
I20260809 01:05:04.647789 28706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:05:04 dist-test-slave-07c2 krb5kdc[28614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237504, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:05:04.650841 28706 init.cc:377] Logged in from keytab as kudu/127.4.82.189@KRBTEST.COM (short username kudu)
I20260809 01:05:04.652289 28706 webserver.cc:460] Webserver started at http://127.4.82.189:35225/ using document root <none> and password file <none>
I20260809 01:05:04.652956 28706 fs_manager.cc:359] Metadata directory not provided
I20260809 01:05:04.653090 28706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:05:04.653419 28706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:05:04.656675 28706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/data/instance:
uuid: "9bb021c6387f4cb287580f0084a4d60c"
format_stamp: "Formatted at 2026-08-09 01:05:04 on dist-test-slave-07c2"
server_key: "2e0fd9862b225ea6e803215fc1c2a587"
server_key_iv: "aa3d8c990895391103a1a5c8b6a6b2f8"
server_key_version: "encryptionkey@0"
I20260809 01:05:04.657462 28706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/wal/instance:
uuid: "9bb021c6387f4cb287580f0084a4d60c"
format_stamp: "Formatted at 2026-08-09 01:05:04 on dist-test-slave-07c2"
server_key: "2e0fd9862b225ea6e803215fc1c2a587"
server_key_iv: "aa3d8c990895391103a1a5c8b6a6b2f8"
server_key_version: "encryptionkey@0"
I20260809 01:05:04.662288 28706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 01:05:04.665568 28722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:05:04.666831 28706 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 01:05:04.667016 28706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/wal
uuid: "9bb021c6387f4cb287580f0084a4d60c"
format_stamp: "Formatted at 2026-08-09 01:05:04 on dist-test-slave-07c2"
server_key: "2e0fd9862b225ea6e803215fc1c2a587"
server_key_iv: "aa3d8c990895391103a1a5c8b6a6b2f8"
server_key_version: "encryptionkey@0"
I20260809 01:05:04.667243 28706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:05:04.686336 28706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:05:04.690112 28706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:05:04.690389 28706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:05:04.708496 28706 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.189:36625
I20260809 01:05:04.708503 28775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.189:36625 every 8 connection(s)
I20260809 01:05:04.709875 28706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/data/info.pb
I20260809 01:05:04.713511 28776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:05:04.715101 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 28706
I20260809 01:05:04.715348 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-1/wal/instance
I20260809 01:05:04.715821 4426 external_mini_cluster.cc:1428] Setting key 0425f3ac0108748cc2290b75ebe88fad
I20260809 01:05:04.720016 28776 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:05:04 dist-test-slave-07c2 krb5kdc[28614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237504, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.189@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:05:04.741765 28701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.722531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43532 (local address 127.4.82.190:38677)
0809 01:05:04.722713 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:04.722718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:04.724073 (+ 1355us) server_negotiation.cc:407] Connection header received
0809 01:05:04.725974 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:04.725980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:04.726076 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:04.726188 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:04.727716 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.728585 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.730091 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.730862 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.731008 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:04.734625 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:04.734634 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:04.734642 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:04.734688 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:04.738403 (+ 3715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:04.739199 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:04.739208 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:04.739214 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:04.739397 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:04.739881 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:04.739889 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:04.739895 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:04.740308 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:04.740471 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:04.741098 (+ 627us) server_negotiation.cc:299] Negotiation successful
0809 01:05:04.741353 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 01:05:04.742298 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.723099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:38677 (local address 127.0.0.1:43532)
0809 01:05:04.723908 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.723945 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:04.725690 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:04.726429 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:04.726437 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:04.726957 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:04.727543 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:04.727560 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.728788 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.728792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:04.729939 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:04.729951 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.731067 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.731072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:04.731270 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:04.732016 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:04.732037 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:04.734383 (+ 2346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:04.738606 (+ 4223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:04.738611 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:04.738632 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:04.739042 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:04.739533 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:04.739540 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:04.739546 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:04.739745 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:04.740438 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:04.740443 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:04.740714 (+ 271us) client_negotiation.cc:769] Sending connection context
0809 01:05:04.740922 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 01:05:04.741278 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":376,"threads_started":1}
I20260809 01:05:04.743898 28776 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.82.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.82.188@KRBTEST.COM" created.
Aug 09 01:05:04 dist-test-slave-07c2 krb5kdc[28614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237504, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.189@KRBTEST.COM for kudu/127.4.82.189@KRBTEST.COM
I20260809 01:05:04.758565 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.744653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:36625 (local address 127.0.0.1:35422)
0809 01:05:04.744855 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.744876 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:04.744982 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:04.745776 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:04.745784 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:04.746277 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:04.746505 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:04.746519 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.747912 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.747920 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:04.749156 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:04.749175 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.749928 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.749932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:04.750052 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:04.750626 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:04.750642 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:04.752881 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:04.756110 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:04.756117 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:04.756123 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:04.756461 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:04.756848 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:04.756854 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:04.756858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:04.756996 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:04.757593 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:04.757602 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:04.757751 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 01:05:04.757899 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 01:05:04.758178 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 01:05:04.758668 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.744772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35422 (local address 127.4.82.189:36625)
0809 01:05:04.745365 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:04.745370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:04.745394 (+ 24us) server_negotiation.cc:407] Connection header received
0809 01:05:04.745432 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:04.745441 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:04.745610 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:04.745727 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:04.746679 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.747763 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.749296 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.749804 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.749896 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:04.753097 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:04.753108 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:04.753120 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:04.753146 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:04.755963 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:04.756609 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:04.756617 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:04.756619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:04.756732 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:04.757099 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:04.757106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:04.757110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:04.757449 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:04.757666 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:04.758076 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 01:05:04.758296 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":318,"threads_started":1}
I20260809 01:05:04.761415 28776 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:05:04.762562 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.762120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:40958)
0809 01:05:04.762308 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.762431 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 01:05:04.762852 28726 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
I20260809 01:05:04.764532 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.764212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:40964)
0809 01:05:04.764364 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.764436 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 01:05:04.764765 28776 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:36533: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.82.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.188.keytab.
Entry for principal kudu/127.4.82.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.188.keytab.
I20260809 01:05:04.786502 4426 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
/tmp/dist-test-taskua6Q0e/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.188:36533
--webserver_interface=127.4.82.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.188.keytab
--principal=kudu/127.4.82.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.82.148:45235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.82.190:38677,127.4.82.189:36625,127.4.82.188:36533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 01:05:04.814081 28776 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } attempt: 1
I20260809 01:05:04.815224 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.814757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:40972)
0809 01:05:04.814962 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.815071 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 01:05:04.816362 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.816016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:40980)
0809 01:05:04.816173 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.816260 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 01:05:04.816573 28776 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:36533: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
I20260809 01:05:04.884821 28776 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } attempt: 2
I20260809 01:05:04.885974 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.885509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:40994)
0809 01:05:04.885726 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.885822 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 01:05:04.887032 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.886713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41008)
0809 01:05:04.886863 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.886928 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 01:05:04.887328 28776 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:36533: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
I20260809 01:05:04.906108 28699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } attempt: 5
Aug 09 01:05:04 dist-test-slave-07c2 krb5kdc[28614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237504, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for kudu/127.4.82.189@KRBTEST.COM
I20260809 01:05:04.918928 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.906904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35424 (local address 127.4.82.189:36625)
0809 01:05:04.907057 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:04.907064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:04.907089 (+ 25us) server_negotiation.cc:407] Connection header received
0809 01:05:04.907330 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:04.907337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:04.907417 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:04.907525 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:04.908333 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.909030 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.910196 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.910627 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.910766 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:04.913836 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:04.913844 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:04.913854 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:04.913885 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:04.916438 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:04.917076 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:04.917084 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:04.917087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:04.917219 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:04.917599 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:04.917604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:04.917608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:04.917875 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:04.918038 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:04.918409 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 01:05:04.918607 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 01:05:04.918951 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.906797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:36625 (local address 127.0.0.1:35424)
0809 01:05:04.907013 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.907032 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:04.907172 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:04.907565 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:04.907571 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:04.907982 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:04.908178 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:04.908187 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.909167 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.909175 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:04.910039 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:04.910049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:04.910769 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:04.910774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:04.910987 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:04.911698 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:04.911714 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:04.913646 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:04.916575 (+ 2929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:04.916581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:04.916585 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:04.916958 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:04.917334 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:04.917339 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:04.917342 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:04.917481 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:04.918002 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:04.918007 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:04.918158 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 01:05:04.918312 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 01:05:04.918581 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 01:05:04.920073 28699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } has no permanent_uuid. Determining permanent_uuid...
I20260809 01:05:04.921155 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.920808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41014)
0809 01:05:04.920956 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.921026 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 01:05:04.922096 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.921766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41022)
0809 01:05:04.921908 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.922006 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 01:05:04.922298 28699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:36533: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
I20260809 01:05:04.973510 28699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } attempt: 1
I20260809 01:05:04.974619 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.974132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41026)
0809 01:05:04.974344 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.974442 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 01:05:04.975667 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.975335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41042)
0809 01:05:04.975491 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.975556 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 01:05:04.975869 28699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:36533: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
I20260809 01:05:04.978543 28776 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } attempt: 3
I20260809 01:05:04.979496 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.979032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41044)
0809 01:05:04.979232 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.979364 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 01:05:04.980582 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:04.980177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41046)
0809 01:05:04.980397 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:04.980472 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260809 01:05:04.980813 28776 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:36533: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
I20260809 01:05:05.044113 28699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } attempt: 2
I20260809 01:05:05.045527 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.044933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41052)
0809 01:05:05.045189 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:05.045305 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 01:05:05.046763 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.046314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41066)
0809 01:05:05.046496 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:05.046588 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 01:05:05.047107 28699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:36533: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
I20260809 01:05:05.124032 28776 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } attempt: 4
I20260809 01:05:05.125437 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.124775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41072)
0809 01:05:05.125065 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:05.125214 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 01:05:05.126695 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.126234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41084)
0809 01:05:05.126414 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:05.126511 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 01:05:05.126950 28776 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:36533: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
W20260809 01:05:05.137681 28785 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 01:05:05.138047 28785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 01:05:05.138178 28785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 01:05:05.138321 28785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 01:05:05.145748 28785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 01:05:05.145855 28785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 01:05:05.145972 28785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 01:05:05.146057 28785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 01:05:05.146163 28785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 01:05:05.155831 28699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } attempt: 3
I20260809 01:05:05.155939 28785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.82.148:45235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.82.190:38677,127.4.82.189:36625,127.4.82.188:36533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/krb5kdc/kudu_127.4.82.188.keytab
--principal=kudu/127.4.82.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.82.188:36533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.82.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 23:55:54 UTC on ccycloud.quasar-bjxnky.root.comops.site
build id 4575725
ASAN enabled
I20260809 01:05:05.157114 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.156452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41100)
0809 01:05:05.156748 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:05.156875 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 01:05:05.157351 28785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 01:05:05.158437 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.157943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41110)
0809 01:05:05.158196 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:05.158292 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260809 01:05:05.158710 28699 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.82.188:36533: Network error: Client connection negotiation failed: client connection to 127.4.82.188:36533: connect: Connection refused (error 111)
I20260809 01:05:05.158988 28785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 01:05:05.173843 28793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:05:05.173905 28796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 01:05:05.174306 28792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 01:05:05.174819 28785 server_base.cc:1034] running on GCE node
I20260809 01:05:05.175697 28785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 01:05:05.177045 28785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 01:05:05.178287 28785 hybrid_clock.cc:648] HybridClock initialized: now 1786237505178189 us; error 70 us; skew 500 ppm
I20260809 01:05:05.178617 28785 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 01:05:05 dist-test-slave-07c2 krb5kdc[28614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237505, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 01:05:05.182297 28785 init.cc:377] Logged in from keytab as kudu/127.4.82.188@KRBTEST.COM (short username kudu)
I20260809 01:05:05.184219 28785 webserver.cc:460] Webserver started at http://127.4.82.188:38445/ using document root <none> and password file <none>
I20260809 01:05:05.185263 28785 fs_manager.cc:359] Metadata directory not provided
I20260809 01:05:05.185410 28785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 01:05:05.185863 28785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 01:05:05.190518 28785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/data/instance:
uuid: "123752c31cab42afb7a7cb2148077e60"
format_stamp: "Formatted at 2026-08-09 01:05:05 on dist-test-slave-07c2"
server_key: "487e20bd593f666eb9a3dde518a3784b"
server_key_iv: "de9210d7de526926589bfdd8712144d1"
server_key_version: "encryptionkey@0"
I20260809 01:05:05.191632 28785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/wal/instance:
uuid: "123752c31cab42afb7a7cb2148077e60"
format_stamp: "Formatted at 2026-08-09 01:05:05 on dist-test-slave-07c2"
server_key: "487e20bd593f666eb9a3dde518a3784b"
server_key_iv: "de9210d7de526926589bfdd8712144d1"
server_key_version: "encryptionkey@0"
I20260809 01:05:05.199218 28785 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260809 01:05:05.204286 28804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 01:05:05.205935 28785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260809 01:05:05.206115 28785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/data,/tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/wal
uuid: "123752c31cab42afb7a7cb2148077e60"
format_stamp: "Formatted at 2026-08-09 01:05:05 on dist-test-slave-07c2"
server_key: "487e20bd593f666eb9a3dde518a3784b"
server_key_iv: "de9210d7de526926589bfdd8712144d1"
server_key_version: "encryptionkey@0"
I20260809 01:05:05.206390 28785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 01:05:05.244987 28785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 01:05:05.251056 28785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 01:05:05.251407 28785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 01:05:05.273018 28856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.82.188:36533 every 8 connection(s)
I20260809 01:05:05.273058 28785 rpc_server.cc:307] RPC server started. Bound to: 127.4.82.188:36533
I20260809 01:05:05.274986 28785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/data/info.pb
I20260809 01:05:05.277839 4426 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu as pid 28785
I20260809 01:05:05.278055 4426 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskua6Q0e/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786237144174446-4426-0/minicluster-data/master-2/wal/instance
I20260809 01:05:05.278717 4426 external_mini_cluster.cc:1428] Setting key 62540a9773154c449389f7cf32895261
I20260809 01:05:05.278885 28857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 01:05:05.290918 28857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:05:05 dist-test-slave-07c2 krb5kdc[28614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.188@KRBTEST.COM
Aug 09 01:05:05 dist-test-slave-07c2 krb5kdc[28614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237505, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.188@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:05:05.320611 28860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.280725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41114 (local address 127.4.82.188:36533)
0809 01:05:05.292232 (+ 11507us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.292247 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.292299 (+ 52us) server_negotiation.cc:407] Connection header received
0809 01:05:05.294461 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.294490 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.295096 (+ 606us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.297999 (+ 2903us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.298623 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.301260 (+ 2637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.303860 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.305232 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.309808 (+ 4576us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.311880 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:05.311890 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:05.311905 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:05.311956 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:05.315695 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.316612 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.316624 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.316639 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.316875 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.317383 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.317389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.317392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.318088 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:05.318401 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.318928 (+ 527us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.319412 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11268,"thread_start_us":11044,"threads_started":1}
I20260809 01:05:05.327842 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.294534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.190:38677 (local address 127.0.0.1:43548)
0809 01:05:05.307932 (+ 13398us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:05.307984 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:05.308127 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:05.310101 (+ 1974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:05.310108 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:05.310847 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:05.311144 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:05.311154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.312719 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.312724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:05.314552 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:05.314568 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.315813 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.315822 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:05.316021 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.316715 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:05.316741 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:05.319870 (+ 3129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:05.324742 (+ 4872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:05.324757 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:05.324766 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:05.325408 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:05.325871 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:05.325878 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:05.325884 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:05.326077 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:05.326708 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:05.326716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:05.326897 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 01:05:05.327074 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 01:05:05.327424 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13146,"thread_start_us":12987,"threads_started":1}
I20260809 01:05:05.327884 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.294403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43548 (local address 127.4.82.190:38677)
0809 01:05:05.309720 (+ 15317us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.309727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.309747 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:05:05.309807 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.309812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.309898 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.310025 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.311371 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.312494 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.314715 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.315610 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.315761 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.320117 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:05.320125 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:05.320129 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:05.320178 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:05.324601 (+ 4423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.325546 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.325553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.325558 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.325735 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.326206 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.326210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.326213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.326560 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:05.326732 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.327203 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.327434 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15168,"thread_start_us":15594,"threads_started":1}
I20260809 01:05:05.329356 28857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:05:05 dist-test-slave-07c2 krb5kdc[28614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.190@KRBTEST.COM
I20260809 01:05:05.336038 28699 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } attempt: 4
Aug 09 01:05:05 dist-test-slave-07c2 krb5kdc[28614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237505, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.188@KRBTEST.COM for kudu/127.4.82.189@KRBTEST.COM
I20260809 01:05:05.345682 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.330407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35432 (local address 127.4.82.189:36625)
0809 01:05:05.330585 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.330591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.330778 (+ 187us) server_negotiation.cc:407] Connection header received
0809 01:05:05.330928 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.330936 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.331029 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.331146 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.331876 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.332701 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.333868 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.334287 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.334419 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.338693 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:05.338705 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:05.338712 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:05.338757 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:05.342455 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.343269 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.343275 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.343280 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.343463 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.343961 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.343966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.343969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.344340 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:05.344479 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.345055 (+ 576us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.345296 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 01:05:05.346725 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.330227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.189:36625 (local address 127.0.0.1:35432)
0809 01:05:05.330672 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:05.330692 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:05.330801 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:05.331193 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:05.331198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:05.331596 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:05.331726 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:05.331734 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.332842 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.332850 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:05.333722 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:05.333733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.334444 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.334450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:05.334646 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.335171 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:05.335188 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:05.338483 (+ 3295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:05.342619 (+ 4136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:05.342626 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:05.342631 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:05.343120 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:05.343626 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:05.343632 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:05.343636 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:05.343817 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:05.344581 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:05.344590 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:05.344757 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 01:05:05.346090 (+ 1333us) client_negotiation.cc:240] Negotiation successful
0809 01:05:05.346372 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260809 01:05:05.347378 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.324105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43558 (local address 127.4.82.190:38677)
0809 01:05:05.324864 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.324868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.324885 (+ 17us) server_negotiation.cc:407] Connection header received
0809 01:05:05.324921 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.324924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.324976 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.325088 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.326590 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.328679 (+ 2089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.330375 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.331064 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.331197 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.337386 (+ 6189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:05.337392 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:05.337397 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:05.337434 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:05.343050 (+ 5616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.343974 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.343979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.343981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.344106 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.344562 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.344566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.344569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.344960 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:05.345228 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.346688 (+ 1460us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.346913 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":399,"threads_started":1}
I20260809 01:05:05.348125 28857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 01:05:05 dist-test-slave-07c2 krb5kdc[28614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237504, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.190@KRBTEST.COM for kudu/127.4.82.188@KRBTEST.COM
I20260809 01:05:05.360935 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.336624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41120)
0809 01:05:05.336968 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:05.336987 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:05.337082 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:05.337762 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:05.337770 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:05.339675 (+ 1905us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:05.339970 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:05.339985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.341677 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.341685 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:05.343598 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:05.343618 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.345060 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.345069 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:05.345353 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.346025 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:05.346054 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:05.351874 (+ 5820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:05.356920 (+ 5046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:05.356928 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:05.356935 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:05.357571 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:05.358135 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:05.358143 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:05.358149 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:05.358351 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:05.359552 (+ 1201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:05.359561 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:05.359832 (+ 271us) client_negotiation.cc:769] Sending connection context
0809 01:05:05.359990 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 01:05:05.360353 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227}
I20260809 01:05:05.362149 28860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.337258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41120 (local address 127.4.82.188:36533)
0809 01:05:05.337443 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.337449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.337466 (+ 17us) server_negotiation.cc:407] Connection header received
0809 01:05:05.337516 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.337522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.337597 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.337700 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.340157 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.341505 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.343838 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.344688 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.344838 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.352132 (+ 7294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:05.352141 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:05.352148 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:05.352196 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:05.356731 (+ 4535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.357779 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.357788 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.357794 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.357977 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.358507 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.358514 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.358521 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.359371 (+ 850us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:05.361366 (+ 1995us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.361516 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.361789 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 01:05:05 dist-test-slave-07c2 krb5kdc[28614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.82.189@KRBTEST.COM
Aug 09 01:05:05 dist-test-slave-07c2 krb5kdc[28614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237505, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.188@KRBTEST.COM for kudu/127.4.82.188@KRBTEST.COM
I20260809 01:05:05.371115 28699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3: Bootstrap starting.
I20260809 01:05:05.372651 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.349554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35434 (local address 127.4.82.189:36625)
0809 01:05:05.350971 (+ 1417us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.350979 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.350999 (+ 20us) server_negotiation.cc:407] Connection header received
0809 01:05:05.351051 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.351056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.351124 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.351233 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.353201 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.354550 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.356842 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.357660 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.357775 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.364587 (+ 6812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:05.364597 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:05.364604 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:05.364646 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:05.368607 (+ 3961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.369554 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.369561 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.369566 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.369759 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.370388 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.370396 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.370403 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.370818 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:05.371275 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.371921 (+ 646us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.372194 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1293}
I20260809 01:05:05.376257 28699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3: Neither blocks nor log segments found. Creating new log.
I20260809 01:05:05.377327 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.349071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41126)
0809 01:05:05.355538 (+ 6467us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:05.355558 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:05.355661 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:05.362567 (+ 6906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:05.362572 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:05.363034 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:05.363256 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:05.363268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.364863 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.364868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:05.366285 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:05.366299 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.367256 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.367262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:05.367489 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.368021 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:05.368046 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:05.370560 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:05.374628 (+ 4068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:05.374632 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:05.374636 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:05.375094 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:05.375568 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:05.375572 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:05.375575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:05.375728 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:05.376390 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:05.376395 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:05.376544 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 01:05:05.376701 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 01:05:05.376983 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6348}
I20260809 01:05:05.377699 28699 log.cc:824] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3: Log is configured to *not* fsync() on all Append() calls
I20260809 01:05:05.378449 28864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.349010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.4.82.188:36533)
0809 01:05:05.362192 (+ 13182us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.362198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.362221 (+ 23us) server_negotiation.cc:407] Connection header received
0809 01:05:05.362268 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.362274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.362345 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.363398 (+ 1053us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.363638 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.364736 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.366447 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.367087 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.367774 (+ 687us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.370788 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:05.370794 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:05.370798 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:05.370833 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:05.374498 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.375263 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.375268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.375271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.375450 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.375854 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.375858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.375862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.376258 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:05.377684 (+ 1426us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.377808 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.377996 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13039,"thread_start_us":13395,"threads_started":1}
I20260809 01:05:05.379551 4426 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 01:05:05.385485 28857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60: Bootstrap starting.
I20260809 01:05:05.383466 28672 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43558
I20260809 01:05:05.390398 28857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60: Neither blocks nor log segments found. Creating new log.
I20260809 01:05:05.392005 28857 log.cc:824] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60: Log is configured to *not* fsync() on all Append() calls
I20260809 01:05:05.394753 28699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3: No bootstrap required, opened a new log
I20260809 01:05:05.395062 28857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60: No bootstrap required, opened a new log
I20260809 01:05:05.404356 28699 raft_consensus.cc:348] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } }
I20260809 01:05:05.404850 28699 raft_consensus.cc:729] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada9ed0eb8cf4c5e900c57f8ebd3b4c3, State: Initialized, Role: FOLLOWER
I20260809 01:05:05.405392 28699 consensus_queue.cc:260] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } }
I20260809 01:05:05.405426 28857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } }
I20260809 01:05:05.405814 28857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 123752c31cab42afb7a7cb2148077e60, State: Initialized, Role: FOLLOWER
I20260809 01:05:05.406292 28857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } }
I20260809 01:05:05.406965 28867 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } } }
I20260809 01:05:05.407951 28867 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:05.408061 28699 sys_catalog.cc:564] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:05:05.408107 28672 raft_consensus.cc:482] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 01:05:05.408631 28672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:05:05.410702 28868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } } }
I20260809 01:05:05.411164 28868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:05.411789 28857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:05:05.413165 28672 raft_consensus.cc:504] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } }
I20260809 01:05:05.415170 28672 leader_election.cc:288] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 9bb021c6387f4cb287580f0084a4d60c (127.4.82.189:36625), 123752c31cab42afb7a7cb2148077e60 (127.4.82.188:36533)
I20260809 01:05:05.417181 28831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "123752c31cab42afb7a7cb2148077e60"
I20260809 01:05:05.417742 28831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60 [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:05:05.422148 28831 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada9ed0eb8cf4c5e900c57f8ebd3b4c3 in term 1.
I20260809 01:05:05.423128 28649 leader_election.cc:302] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 123752c31cab42afb7a7cb2148077e60, ada9ed0eb8cf4c5e900c57f8ebd3b4c3; no voters:
I20260809 01:05:05.423691 28867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 01:05:05.426178 28776 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } attempt: 5
W20260809 01:05:05.437371 28651 tablet.cc:2369] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:05:05.437604 28867 raft_consensus.cc:686] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [term 1 LEADER]: Becoming Leader. State: Replica: ada9ed0eb8cf4c5e900c57f8ebd3b4c3, State: Running, Role: LEADER
Aug 09 01:05:05 dist-test-slave-07c2 krb5kdc[28614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786237504, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.82.189@KRBTEST.COM for kudu/127.4.82.188@KRBTEST.COM
I20260809 01:05:05.438794 28867 consensus_queue.cc:237] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } }
I20260809 01:05:05.445554 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.426930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.82.188:36533 (local address 127.0.0.1:41136)
0809 01:05:05.427374 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 01:05:05.427397 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 01:05:05.427508 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 01:05:05.428457 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 01:05:05.428465 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 01:05:05.429127 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0809 01:05:05.429387 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:05.429404 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.431137 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.431145 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:05.432982 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 01:05:05.433001 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.434373 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.434382 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 01:05:05.434665 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.435373 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 01:05:05.435406 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 01:05:05.438563 (+ 3157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 01:05:05.442507 (+ 3944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:05.442512 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:05.442515 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:05.442995 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:05.443458 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 01:05:05.443462 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 01:05:05.443465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 01:05:05.443631 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 01:05:05.444347 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 01:05:05.444355 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 01:05:05.444524 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 01:05:05.444675 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 01:05:05.445021 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277}
I20260809 01:05:05.446703 28864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.427722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41136 (local address 127.4.82.188:36533)
0809 01:05:05.427879 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.427888 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.427911 (+ 23us) server_negotiation.cc:407] Connection header received
0809 01:05:05.427976 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.427983 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.428121 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.428231 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.429592 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.430938 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.433202 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.433988 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.434146 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.438778 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:05.438788 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:05.438793 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:05.438830 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:05.442334 (+ 3504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.443147 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.443153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.443156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.443322 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.443761 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.443767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.443771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.444135 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:05.444279 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.445945 (+ 1666us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.446256 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 01:05:05.455598 28776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c: Bootstrap starting.
I20260809 01:05:05.467181 28776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c: Neither blocks nor log segments found. Creating new log.
I20260809 01:05:05.470072 28878 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ada9ed0eb8cf4c5e900c57f8ebd3b4c3. Latest consensus state: current_term: 1 leader_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } } }
I20260809 01:05:05.471112 28878 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [sys.catalog]: This master's current role is: LEADER
I20260809 01:05:05.474133 28776 log.cc:824] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c: Log is configured to *not* fsync() on all Append() calls
I20260809 01:05:05.475692 28894 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 01:05:05.475804 28892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 01:05:05.478453 28785 master_runner.cc:186] Error getting master registration for 127.4.82.190:38677: OK, instance_id { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" instance_seqno: 1786237504198164 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 01:05:05.489063 28785 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:05:05.494302 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.474839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43564 (local address 127.4.82.190:38677)
0809 01:05:05.475030 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.475039 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.476498 (+ 1459us) server_negotiation.cc:407] Connection header received
0809 01:05:05.476793 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.476815 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.476887 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.476960 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.479131 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.480260 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.482360 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.483041 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.483167 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.486892 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:05.486898 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:05.486903 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:05.486937 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:05.490638 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.491503 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.491509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.491512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.491698 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.492185 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.492190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.492193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.492482 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:05.492692 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.493683 (+ 991us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.493941 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 01:05:05.504040 28810 tablet.cc:2369] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 01:05:05.504598 28776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c: No bootstrap required, opened a new log
I20260809 01:05:05.512426 28864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.489587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41144 (local address 127.4.82.188:36533)
0809 01:05:05.489770 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.489775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.496143 (+ 6368us) server_negotiation.cc:407] Connection header received
0809 01:05:05.496726 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.496734 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.496835 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.496905 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.498543 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.499746 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.501729 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.502359 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.502458 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.505448 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:05.505457 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:05.505462 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:05.505502 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:05.509004 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.509892 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.509896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.509899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.510073 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.510546 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.510551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.510555 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.510874 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:05.511154 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.511796 (+ 642us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.512009 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 01:05:05.513345 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.474325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35448 (local address 127.4.82.189:36625)
0809 01:05:05.474550 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.474559 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.475069 (+ 510us) server_negotiation.cc:407] Connection header received
0809 01:05:05.475605 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.475615 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.475735 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.475836 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.481622 (+ 5786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.483201 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.488473 (+ 5272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.489500 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.489650 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.497631 (+ 7981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 01:05:05.497643 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 01:05:05.497651 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 01:05:05.497716 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 01:05:05.503516 (+ 5800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.507107 (+ 3591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.507117 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.507124 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.507529 (+ 405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 01:05:05.508663 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 01:05:05.508680 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 01:05:05.508687 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 01:05:05.509707 (+ 1020us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 01:05:05.510293 (+ 586us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.511839 (+ 1546us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.512403 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 01:05:05.515028 28892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 01:05:05.523347 28831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60 [term 1 FOLLOWER]: Refusing update from remote peer ada9ed0eb8cf4c5e900c57f8ebd3b4c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:05:05.529764 28776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } }
I20260809 01:05:05.530125 28776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb021c6387f4cb287580f0084a4d60c, State: Initialized, Role: FOLLOWER
I20260809 01:05:05.530632 28776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } }
I20260809 01:05:05.533203 28867 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:05:05.538702 28904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } } }
I20260809 01:05:05.539120 28904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:05.540874 28776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 01:05:05.541550 28749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c [term 0 FOLLOWER]: Advancing to term 1
I20260809 01:05:05.543485 28750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9bb021c6387f4cb287580f0084a4d60c"
I20260809 01:05:05.552596 28749 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c [term 1 FOLLOWER]: Refusing update from remote peer ada9ed0eb8cf4c5e900c57f8ebd3b4c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 01:05:05.553577 28867 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 01:05:05.568274 28750 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ada9ed0eb8cf4c5e900c57f8ebd3b4c3 in term 1.
I20260809 01:05:05.570832 28868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ada9ed0eb8cf4c5e900c57f8ebd3b4c3. Latest consensus state: current_term: 1 leader_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } } }
I20260809 01:05:05.571203 28868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:05.591724 28867 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } } }
I20260809 01:05:05.592367 28867 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [sys.catalog]: This master's current role is: LEADER
I20260809 01:05:05.596016 28892 catalog_manager.cc:1347] Generated new cluster ID: f67886a9f7ec4e6a956e889f9e3df291
I20260809 01:05:05.596127 28892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 01:05:05.612929 28868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } } }
I20260809 01:05:05.613325 28868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:05.558084 28628 master_runner.cc:186] Error getting master registration for 127.4.82.189:36625: OK, instance_id { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" instance_seqno: 1786237504703312 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 01:05:05.619791 28628 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 01:05:05.634979 28904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ada9ed0eb8cf4c5e900c57f8ebd3b4c3. Latest consensus state: current_term: 1 leader_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } } }
I20260809 01:05:05.635329 28904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:05.636878 28892 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 01:05:05.637646 28878 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } } }
I20260809 01:05:05.638002 28878 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3 [sys.catalog]: This master's current role is: LEADER
I20260809 01:05:05.638859 28892 catalog_manager.cc:1504] Loading token signing keys...
I20260809 01:05:05.658659 28892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ada9ed0eb8cf4c5e900c57f8ebd3b4c3: Generated new TSK 0
I20260809 01:05:05.660075 28904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ada9ed0eb8cf4c5e900c57f8ebd3b4c3" member_type: VOTER last_known_addr { host: "127.4.82.190" port: 38677 } } peers { permanent_uuid: "9bb021c6387f4cb287580f0084a4d60c" member_type: VOTER last_known_addr { host: "127.4.82.189" port: 36625 } } peers { permanent_uuid: "123752c31cab42afb7a7cb2148077e60" member_type: VOTER last_known_addr { host: "127.4.82.188" port: 36533 } } }
I20260809 01:05:05.660351 28904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 01:05:05.660776 28892 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 01:05:05.670970 28925 catalog_manager.cc:1259] Loaded cluster ID: f67886a9f7ec4e6a956e889f9e3df291
I20260809 01:05:05.671147 28925 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c: loading cluster ID for follower catalog manager: success
I20260809 01:05:05.675072 28925 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c: acquiring CA information for follower catalog manager: success
I20260809 01:05:05.677556 28925 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9bb021c6387f4cb287580f0084a4d60c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 01:05:05.762033 28863 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:05.745665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43566 (local address 127.4.82.190:38677)
0809 01:05:05.745862 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.745869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.759196 (+ 13327us) server_negotiation.cc:407] Connection header received
0809 01:05:05.760246 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.760254 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.760368 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.760444 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.761729 (+ 1285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43566: BlockingRecv error: recv got EOF from 127.0.0.1:43566 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 01:05:05.764767 28864 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:05.751462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41150 (local address 127.4.82.188:36533)
0809 01:05:05.751636 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.751643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.763245 (+ 11602us) server_negotiation.cc:407] Connection header received
0809 01:05:05.763457 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.763463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.763544 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.763636 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.764533 (+ 897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41150: BlockingRecv error: recv got EOF from 127.0.0.1:41150 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 01:05:05.766237 28781 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:05.745877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35458 (local address 127.4.82.189:36625)
0809 01:05:05.746082 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.746105 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.763077 (+ 16972us) server_negotiation.cc:407] Connection header received
0809 01:05:05.764668 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.764674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.764775 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.764854 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.765986 (+ 1132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35458: BlockingRecv error: recv got EOF from 127.0.0.1:35458 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 01:05:05.777606 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.769900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35472 (local address 127.4.82.189:36625)
0809 01:05:05.770139 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.770144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.770603 (+ 459us) server_negotiation.cc:407] Connection header received
0809 01:05:05.770862 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.770869 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.771063 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.771228 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:05.771748 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.772883 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.774861 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.775554 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.775696 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.776259 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:05.776943 (+ 684us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:05.777117 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.777198 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.777287 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
W20260809 01:05:05.786434 28864 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:05.784375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41154 (local address 127.4.82.188:36533)
0809 01:05:05.784542 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.784548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.785031 (+ 483us) server_negotiation.cc:407] Connection header received
0809 01:05:05.785232 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.785238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.785355 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.785462 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.786222 (+ 760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41154: BlockingRecv error: recv got EOF from 127.0.0.1:41154 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 01:05:05.799638 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.793014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35484 (local address 127.4.82.189:36625)
0809 01:05:05.793197 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.793203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.793636 (+ 433us) server_negotiation.cc:407] Connection header received
0809 01:05:05.793763 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.793768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.793860 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.793952 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:05.794415 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.795558 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.797576 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.798255 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.798379 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.798830 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:05.798965 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:05.799067 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.799227 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.799336 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 01:05:05.808485 28864 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:05.806436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41166 (local address 127.4.82.188:36533)
0809 01:05:05.806594 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.806600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.807101 (+ 501us) server_negotiation.cc:407] Connection header received
0809 01:05:05.807271 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.807277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.807386 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.807492 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.808267 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41166: BlockingRecv error: recv got EOF from 127.0.0.1:41166 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 01:05:05.825911 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.819140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35486 (local address 127.4.82.189:36625)
0809 01:05:05.819328 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.819334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.819829 (+ 495us) server_negotiation.cc:407] Connection header received
0809 01:05:05.819983 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.819988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.820067 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.820208 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:05.820693 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.821823 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.823793 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.824511 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.824669 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.825110 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:05.825227 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:05.825313 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.825477 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.825559 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 01:05:05.839095 28864 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:05.837027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.4.82.188:36533)
0809 01:05:05.837198 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.837204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.837702 (+ 498us) server_negotiation.cc:407] Connection header received
0809 01:05:05.837859 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.837864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.837957 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.838061 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.838851 (+ 790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41172: BlockingRecv error: recv got EOF from 127.0.0.1:41172 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 01:05:05.853361 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.846708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35500 (local address 127.4.82.189:36625)
0809 01:05:05.846917 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.846924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.847417 (+ 493us) server_negotiation.cc:407] Connection header received
0809 01:05:05.847582 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.847588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.847681 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.847776 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:05.848225 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.849371 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.851359 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.852060 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.852188 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.852637 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:05.852761 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:05.852854 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.853016 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.853093 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
W20260809 01:05:05.863536 28864 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:05.861042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41182 (local address 127.4.82.188:36533)
0809 01:05:05.861309 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.861317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.861853 (+ 536us) server_negotiation.cc:407] Connection header received
0809 01:05:05.862123 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.862128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.862219 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.862289 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.863318 (+ 1029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41182: BlockingRecv error: recv got EOF from 127.0.0.1:41182 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
I20260809 01:05:05.884728 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.877277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35508 (local address 127.4.82.189:36625)
0809 01:05:05.877505 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.877511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.878389 (+ 878us) server_negotiation.cc:407] Connection header received
0809 01:05:05.878613 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.878619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.878717 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.878819 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:05.879305 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.880549 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.882548 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.883252 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.883434 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.883878 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:05.884014 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:05.884109 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.884282 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.884371 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 01:05:05.896437 28864 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:05.894125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41188 (local address 127.4.82.188:36533)
0809 01:05:05.894305 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.894311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.894837 (+ 526us) server_negotiation.cc:407] Connection header received
0809 01:05:05.895023 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.895029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.895138 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.895246 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.896185 (+ 939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41188: BlockingRecv error: recv got EOF from 127.0.0.1:41188 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 01:05:05.924649 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.917383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35510 (local address 127.4.82.189:36625)
0809 01:05:05.917588 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.917605 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.918158 (+ 553us) server_negotiation.cc:407] Connection header received
0809 01:05:05.918369 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.918374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.918488 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.918595 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:05.919093 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.920390 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.922421 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.923130 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.923268 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.923818 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:05.923999 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:05.924105 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.924262 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.924359 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260809 01:05:05.935387 28864 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:05.933100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41196 (local address 127.4.82.188:36533)
0809 01:05:05.933305 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.933312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.933865 (+ 553us) server_negotiation.cc:407] Connection header received
0809 01:05:05.934058 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.934067 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.934167 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.934280 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.935148 (+ 868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41196: BlockingRecv error: recv got EOF from 127.0.0.1:41196 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 01:05:05.978902 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:05.971535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35514 (local address 127.4.82.189:36625)
0809 01:05:05.971774 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.971780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.972343 (+ 563us) server_negotiation.cc:407] Connection header received
0809 01:05:05.972576 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.972583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.972702 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.972805 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:05.973328 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.974468 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.976516 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:05.977335 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:05.977491 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:05.977987 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:05.978128 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:05.978227 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:05.978410 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 01:05:05.978493 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 01:05:05.988766 28864 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:05.986346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41212 (local address 127.4.82.188:36533)
0809 01:05:05.986544 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:05.986551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:05.987093 (+ 542us) server_negotiation.cc:407] Connection header received
0809 01:05:05.987319 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:05.987326 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:05.987437 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:05.987547 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:05.988503 (+ 956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41212: BlockingRecv error: recv got EOF from 127.0.0.1:41212 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 01:05:06.063910 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:06.057151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35528 (local address 127.4.82.189:36625)
0809 01:05:06.057352 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:06.057359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:06.057907 (+ 548us) server_negotiation.cc:407] Connection header received
0809 01:05:06.058101 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:06.058107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:06.058194 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:06.058293 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:06.058742 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.059867 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.061733 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.062474 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.062604 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:06.063062 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:06.063203 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:06.063314 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:06.063471 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 01:05:06.063558 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 01:05:06.073217 28864 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:06.071037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41222 (local address 127.4.82.188:36533)
0809 01:05:06.071240 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:06.071245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:06.071829 (+ 584us) server_negotiation.cc:407] Connection header received
0809 01:05:06.071985 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:06.071991 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:06.072093 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:06.072207 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:06.073005 (+ 798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41222: BlockingRecv error: recv got EOF from 127.0.0.1:41222 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 01:05:06.209451 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:06.204406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35542 (local address 127.4.82.189:36625)
0809 01:05:06.204652 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:06.204661 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:06.204864 (+ 203us) server_negotiation.cc:407] Connection header received
0809 01:05:06.205098 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:06.205110 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:06.205180 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:06.205278 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:06.205723 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.206413 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.207751 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.208158 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.208289 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:06.208686 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:06.208826 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:06.208939 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:06.209088 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 01:05:06.209152 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260809 01:05:06.216393 28864 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 01:05:06.214660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41232 (local address 127.4.82.188:36533)
0809 01:05:06.214818 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:06.214823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:06.215211 (+ 388us) server_negotiation.cc:407] Connection header received
0809 01:05:06.215431 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:06.215434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:06.215503 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:06.215638 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 01:05:06.216212 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41232: BlockingRecv error: recv got EOF from 127.0.0.1:41232 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 01:05:06.480805 28894 catalog_manager.cc:1259] Loaded cluster ID: f67886a9f7ec4e6a956e889f9e3df291
I20260809 01:05:06.480944 28894 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60: loading cluster ID for follower catalog manager: success
I20260809 01:05:06.482059 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:06.476688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35546 (local address 127.4.82.189:36625)
0809 01:05:06.476922 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:06.476929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:06.477223 (+ 294us) server_negotiation.cc:407] Connection header received
0809 01:05:06.477453 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:06.477457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:06.477525 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:06.477630 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:06.478113 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.478848 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.480260 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.480657 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.480799 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:06.481328 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:06.481427 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:06.481528 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:06.481689 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 01:05:06.481768 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 01:05:06.484508 28894 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60: acquiring CA information for follower catalog manager: success
I20260809 01:05:06.486883 28894 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 123752c31cab42afb7a7cb2148077e60: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 01:05:06.494002 28864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:06.488022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.4.82.188:36533)
0809 01:05:06.488189 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:06.488196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:06.488418 (+ 222us) server_negotiation.cc:407] Connection header received
0809 01:05:06.488611 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:06.488615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:06.488762 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:06.488850 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:06.489320 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.490018 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.491903 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.492325 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.492415 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:06.492978 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:06.493375 (+ 397us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:06.493470 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:06.493637 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 01:05:06.493706 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 01:05:06.533965 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:06.500496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43578 (local address 127.4.82.190:38677)
0809 01:05:06.524680 (+ 24184us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:06.524688 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:06.525729 (+ 1041us) server_negotiation.cc:407] Connection header received
0809 01:05:06.525880 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:06.525886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:06.526111 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:06.526303 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:06.527146 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.528475 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.530802 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.531445 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.531582 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:06.532593 (+ 1011us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:06.533142 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:06.533255 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:06.533529 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 01:05:06.533617 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24014,"thread_start_us":24210,"threads_started":1}
I20260809 01:05:06.542900 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:06.500748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35550 (local address 127.4.82.189:36625)
0809 01:05:06.500917 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:06.500922 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:06.534641 (+ 33719us) server_negotiation.cc:407] Connection header received
0809 01:05:06.535050 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:06.535058 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:06.535150 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:06.535218 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:06.535920 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.536985 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.540168 (+ 3183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.540844 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.540952 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:06.542021 (+ 1069us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:06.542163 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:06.542249 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:06.542490 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 01:05:06.542576 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 01:05:06.544214 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 28628
I20260809 01:05:06.545166 28693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:05:06.547299 28864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 01:05:06.499999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41254 (local address 127.4.82.188:36533)
0809 01:05:06.500215 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 01:05:06.500223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 01:05:06.539550 (+ 39327us) server_negotiation.cc:407] Connection header received
0809 01:05:06.540172 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 01:05:06.540177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 01:05:06.540268 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 01:05:06.540369 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 01:05:06.541278 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.542403 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.544646 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 01:05:06.545359 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 01:05:06.545521 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 01:05:06.546333 (+ 812us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 01:05:06.546502 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 01:05:06.546572 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 01:05:06.546827 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 01:05:06.546930 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 01:05:06.664942 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 28628
W20260809 01:05:06.700157 28726 connection.cc:535] server connection from 127.0.0.1:35424 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 01:05:06.700824 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 28706
I20260809 01:05:06.702008 28771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:05:06.810694 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 28706
I20260809 01:05:06.833774 4426 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu pid 28785
I20260809 01:05:06.834885 28852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 01:05:06.950433 4426 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskua6Q0e/build/asan/bin/kudu with pid 28785
2026-08-09T01:05:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4627 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9603 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (362743 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST